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Executive  Summary 

This  report  describes  the  results  and  recommendations  derived 
from  an  extensive  survey  of  existing  human  performance  models  and 
modelling  approaches  applicable  to  the  design  and  evaluation  of 
large-scale  command  and  control  systems.  The  focus  is  on  models 
derived  from  a control-  and  decision-theoretic  framework,  the 
modelling  literature  in  human  information  processing,  and  the 
collection  of  models  and  data-bank  formulations  originally  derived 
from  the  reliability  and  network-simulation  literature. 

The  most,  successful  modelling  efforts  seem  to  have  grown 
out  of  situations  where  formal  models  of  the  task  environment  are 
well  developed,  such  as  in  feedback  control  tasks,  detection 
tasks,  and  well-defined  probabilistic  decision-making  tasks. 
Further,  in  these  areas,  the  most  successful  of  these  models 
arise  when  the  researcher  can  express  formal  criteria  of  optimal 
performance  as  reflected  in  the  optimal  control  formulation  of 
manual  control  or  the  ideal  observer  in  target  detection  and 
recognition  tasks.  These  observations  emphasize  the  importance 
to  successful  modelling  efforts  of  being  able  to  express  the 
goals  and  success  criteria  used  by  human  operators  in  formal 
quantitative  terms.  One  difficulty  for  modelling  behavior  in 
more  complex  procedural  tasks  arises  from  the  inherently  multi- 
dimensional, multi-level,  time-varying  array  of  criteria  and 
strategies  that  an  operator  applies  in  accomplishing  these  tasks. 


It  is  interesting  to  note  that  the  optimal  control  model 
and  those  information-processing  models  derived  from  a decision- 
theoretic  framework  are  mutually  compatible;  this  suggests  the 
possibility  of  integrating  and  generalizing  them  to  provide  n 
single  modelling  framework  that  could  be  applied  to  vehicle 
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control,  supervisory  monitoring,  surveillance,  signal  identifica- 
tion, and  decision  making,  all  of  which  are  tasks  of  major 
interest  in  military  system  design  and  evaluation. 

In  the  area  of  intellectual  performance,  however,  modelling 
efforts  have  not  produced  practically  useful  results,  either  in 
areas  where  an  explicit  algorithm  might  be  specified  or  with 
respect  to  general  problem-solving  performance,  where  a wide  range 
of  performance  strategies  are  available.  To  represent  these  kinds 
of  performance  will  require  either  structuring  the  problem  so 
that  results  are  not  sensitive  to  differences  in  strategy  or 
resorting  to  atheoretic  representations  derived  from  empirical 
measurements  obtained  in  the  specific  task  context. 

In  addition  to  the  substantive  reviews  of  modelling  approaches, 
several  methodological  issues  have  been  identified: 

(1)  The  problem  of  validation  of  models  of  the  scope  considered 
here  is  a difficult  one,  requiring  further  research. 

(2)  Models  exist  at  many  different  levels  of  specificity.  A 
substantive  issue  concerns  the  identification  of  the  level 
of  specificity  at  which  to  define  a model  in  relation  to 
the  goals  of  system  performance  prediction  desired.  Depending 
on  the  level  of  specificity  that  is  appropriate,  one  must 
consider  whether  to  take  a top-down  or  a bottom-up  approach. 

A top-down  approach  begins  with  goals  of  performance 
prediction  and  represents  performance  only  down  to  level 
required  to  meet  these  goals.  A bottom-up  approach  begins 
by  defining  the  elemental  components  of  human  performance 
and  synthesizes  them  inLo  a model  that  predicts  the  desired 
aspects  of  performance. 
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(3)  Bottom-up  approaches  to  modelling  continually  face  the  issue 
of  how  to  combine  sub-task  or  task  models  into  higher-level 
structures  in  such  a way  that  the  potential  interactions 
resulting  from  their  combination  are  accounted  for.  Additive 
combinations  of  component  response  times  and  multiplicative 
combinations  of  response  accuracies  are  frequently  not  valid, 
and  their  applicability  must  be  evaluated  in  each  new  synthesis. 

(4)  The  current  state  of  theory  and  understanding  of  human 
Performance  is  inadequate  to  represent  many  kinds  of 
behavior  observed  in  real  task  situations.  The  model 
developer  is  left  with  many  arbitrary  parameters  that  must 
be  defined  on  the  basis  of  observed  performance.  When  the 
number  of  such  free  parameters  approaches  the  number  of 
performance  measures  to  be  predicted,  the  predictive  power 
of  the  model  is  severely  compromised. 

On  the  basis  of  our  review,  recommendations  are  introduced 
for  further  research  and  development  of  large  scale  systems 
modelling  efforts.  These  recommendations  include:  ' 

(1)  Development  of  a test-bed  facility  in  which  to  evaluate 
alternative  model  formulations  of  common  task  environments  and  to 
conduct  empirical  validation  studies  to  compare  model  predictions 
with  actual  human  performance. 

(2)  Methodological  research  on  (a)  the  implications  of 
combining  sub-task  or  information  processing  component  models  on 
system  performance  in  the  aggregate,  (b)  validati  >n  of  large  scale 
simulation  models,  and  (c)  development  of  guidelines  for  the 
acceptable  number  of  free  parameters  in  useful  predictive  models. 
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(3)  Recommended  furl  her  m >del  development  in  topical  areas 
of  high  priority  for  representation  of  command  and  control  system 
Two  such  areas  are  supervisory  control  and  monitoring  and 

the  prediction  of  team  performance  on  the  basis  of  performance  of 
individual  team  members. 

(4)  Advancing  the  state-of-the-art  with  respect  to  the 
specific  modelling  approaches  discussed  in  the  body  of  the  report 

From  an  overall  perspective,  we  believe  that  integrative 
models  of  human  performance  compatible  with  the  requirements  for 
representing  command  and  control  system  performance  do  not  exist 
at  the  present  time.  What  is  available  is  a collection  of 
component  models  and  modelling  principles  formulated  in  a variety 
of  frameworks,  which  might  be  drawn  together  to  build  an  eclectic 
model  for  particular  task  situations  of  interest.  On  the  basis 
of  our  present  level  of  understanding,  assembly  of  the  components 
will  call  for  substantial  effort  and  is  likely  to  require  many 
assumptions  about  particular  aspects  of  performance.  If  one  is 
to  have  confidence  in  the  product  generated  in  this  way,  several 
iterative  validation  steps  will  be  required. 
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1.0  INTRODUCTION 

The  Systems  Research  Branch  of  the  Human  Engineering  Division 
of  the  Aerospace  Medical  Research  Division  (SRB)  has  undertaken  a 
long-term  program  to  develop  and  exploit  simulation  and  modelling 
technology  in  the  design  and  evaluation  of  large-scale  systems. 

In  support  of  this  goal,  BBN  has  initiated  a three-year  program  of 
research  to  examine  existing  human  performance  models  and  approaches 
to  modelling  performance  and  to  conduct  studies  exploring  potential 
extensions  of  this  technology. 

Our  goal  is  to  develop  the  beginnings  of  a handbook-like 
document  that  would  be  useful  to  systems  designers  and  analysts 
embarking  on  a systems  modelling  effort.  Our  research  program 
assumes  that  computerized  models  of  the  system  under  consideration 
are  to  be  constructed,  and  that  these  models  will  take  into  account 
the  behavior  of  che  human  operators  interacting  with  the  system. 

By  exercising  these  models,  systems  engineers  wish  to  predict  the 
effects  of  changes  in  system  parameters  before  proceeding  to  full- 
scale  simulation  efforts  and  the  operational  evaluation  of  prototype 
systems . 

In  the  first  year  of  this  program,  we  have  focused  on  the 
review  of  potentially  relevant  models  and  on  the  identification 
of  issues  in  model  development  and  application  that  may  have  an 
important  impact  on  the  success  of  such  modelling  efforts.  Tn 
the  second  and  third  years,  we  propose  to  conduct  modelling  studio:; 
in  collaboration  with  personnel  of  SRB  to  explore  quantitatively 
and  in  detail  several  key  issues  in  model  dev.  loptnen!  . 
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This  document  represents  the  final  report  of  the  first  year 
of  effort. 

1 . 1 Background 

The  current  state  of  methodology  for  the  development  of 
large-scale,  multi-man  command  and  control  systems,  particularly 
tactical  systems, is  largely  an  art.  The  designers  formulate 
goals  and  system  objectives.  They  examine  the  available  technology 
and  produce  a conceptual  design  that  specifies  the  roles  of 
hardware,  software,  and  human  resources.  They  then  proceed  to  build 
either  a manned  simulation  of  the  basic  design  or  a prototype 
system  that  can  serve  as  a test  bed  for  evaluating  the  approach  and 
for  exploring  alternative  implementations  of  the  basic  approach. 

HED  has  pioneered  in  the  application  of  the  simulation  approach  for 
three  different  Air  Force  Systems,  BUTC,  AWACS,  and  Remotely 
Piloted  Vehicle/Drone  (RPVj  Control,  and  has  demonstrated  the 
feasibility  and  cost  effectiveness  of  preceding  the  prototype 
development  phase  by  a simulation  phase  in  which  formal  experiments 
of  differing  implementations  may  be  undertaken. 

HED  is  also  looking  ahead  to  the  possibility  of  further 
efficiencies  in  system  development  through  the  use  of  full  computer 
modelling  of  the  hardware,  software,  and  human  performance  components 
of  new  systems  that  would  permit  design  trade-off  studies  to  be 
undertaken  in  advance  of  full-scale  manned  simulation. 

To  this  end  they  have  developed  the  SAINT  simulat  ion  language 
(Wortman,  ot.nl.,  1974),  a dialect  of  C.ASP  that  has  been  specifically 
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operating  system  that  permits  easy  translation  from  a flow  chart 
representation  of  the  system  to  a computer-codec!  model. 

The  basic  approach  to  modelling  represented  in  SAINT  and 
other  event-oriented  modelling  languages  is  to  analyse  the  system 
flow  chart  into  a series  of  processes  related  to  each  other 
through  a contingently-branching  structure.  For  each  process,  the* 
necessary  initial  conditions  for  process  activation  are  specified. 
Then  a specified  distribution  of  completion  times  and  a 
probability  of  successful  completion  is  assigned  to  the 
process.  These  parameters  of  each  process  may  also  be  modified 
contingently  while  a simulation  is  in  progress.  Each  time  the 
model  is  exercised,  a particular  sample  of  each  random  variable  is 
chosen  and  the  contingent  effects  are  assessed  so  that  a unique  trac 
through  the  multi-path  flow  chart  is  completed.  Aggregated  values 
for  completion  time  and  probability  of  completion  are  then 
derived  from  repeated  runs  of  the  simulation.  The  model  is 
exercised  sufficiently  to  accumulate  reliable  statistics 
for  the  major  dependent  variables  under  study,  and  it  is 
these  statistics  that  are  the  output,  of  the  modal. 

The  usefulness  of  SAINT  and  of  the  modelling  approach  to 
system  design  have  been  explored  in  two  major  modelling  studies. 

A model  of  a 3-man  AW ACS  surveillance  system  was  built  using  SAINT 
in  piirallel  with  the  comparable  manned  simulation  of  the  system 
(Mills,  1976).  It  was  shown  that  a verified  model  could  be 
obtained  to  describe  system  performance  for  the  specific  condition 
under  study.  However,  a great  deal  of  effort  and  an  estimated  50 
iterations  of  the  model  form  and  parameters  were  required  to  debug 
it  and  to  produce  sat.  isi  ictory  matches  to  the  simulation  data. 
Further,  it  was  shown  that  the  model  did  not  generalize  very  well 
to  predict  group  per  formancv  when  pa  ram  .'tern  representing  the 
performance  of  individual  operators  act  i ng  in  isolation  wore  us  '.1 . 
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In  a second  application,  an  upgraded  version  of  the  SAINT 
language  was  used  to  model  the  performance  of  the  RPV/Drone  Control 
System  for  which  several  manned  simulation  studies  have  been  com; >1 ete< 
(Uortman , et.al  . , 197  j)  . With  a similar  large  of  fort  to  dehug  and  tune 
the  parameters  of  the  model,  a successful  representation  of  the  major 
system  performance  variables  of  RPV  System  Study  II  was  obtained 
(Hills,  Bachert,  and  Auroe,  1975) . Again,  although  generalization 
was  not  specifically  tested,  examination  of  the  model  suggests 
that  it  would  not  be  particularly  useful  for  design  trade-off 
studies  because  the  model  was  not  effectively  modularized  to  allow 
extension  to  new  conditions  without  major  adjustments  in  model 
structure  and  parameter  settings.  It  was  necessary  for  the  model 
implementors  to  mate  too  literal  a representation  of  particular 
features  of  the  situation  in  order  to  achieve  the  successful 
representation  of  overall  performance.  Said  another  way,  the 
processes  modelled  were  tasks  that  included  components  of  both 
human  and  machine  performance.  Cheinging  systems  variables  will 
produce  changes  that  interact  with  human  performance  in  now  ways 
that  are  not  easily  separable  in  the  model.  Whether  these 
difficulties  are  inherent  in  the  modelling  approach  to  describing 
large  scale  systems  or  are  only  limitations  of  the  specific  cases 
undertaken  thus  far  remains  to  be  seen.  It  does  seem  clear, 
however,  that  further  progress  in  such  modelling  will  require 
further  development  of  modelling  methodology  and  validation  with 
specific  attention  to  the  representation  of  human  performance  in 
such  systems  in  a way  that  has  the  potential  for  generalization 
to  answer  real  systems  questions.  The  models  for  human  performance 
need  to  be  structured  to  be  compatible  wi  ‘ h the  representation  of. 
the  major  systems  variables  of  interest  to  the  systems  design 
engineer  and  not  simply  embedded  implicitly  in  models  for  the 
major  tasks  invr  ' ved . 
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Other  laboratories  (for  example,  Sandia,  Naval  7vir  Develop- 
ment Center , and  Applied  Psychological  Services)  have  been  developing 
large-scale  modelling  methodologies  with  varying  degrees  of  similarity 
to  the  work  in  the  Air  Force.  However,  few  (if  any) 
of  these  efforts  have  had  available  resources  equivalent  to  the 
HED  multi-man  simulation  facility  in  which  models  may  be 
explicitly  compared  and  validated  against  the  results  of 
real-time/  manned  simulations.  Thus  the  Air  Force  has  a unique 
opportunity  to  advance  the  state-of-the-art  in  system  modelling 
technology . 

It  is  to  this  end  that  Bolt  Beranek  and  Newman  has  undertaken 

this  program  of  research.  During  the  first  year  we  have  examined 

the  rather  extensive  psychological  and  systems  literature  on 

models  of  human  performance.  Our  sources  have  included: 

Psychological  Bulletin 

Psychological  Review 

Journal  of  Experimental  Psychology 

Journal  of  Mathematical  Psychology 

Human  Factors 

IEEE  Transactions 

International  Journal  of  Man-Machine  Studies 
Operations  Research 

Proceedings  of  Annual  Conferences  on  Manual  Control 
JANAIR  Reports 

Technical  Reports  on  Air  Traffic  Control 

Books  and  Monographs  on  Information  Processing  Models 

Miscellaneous  Contractor  and  Government  Reports 

We  have  struggled  repeatedly  with  questions  concerning  the 
scope  of  models  appropriate  to  this  review  and  evaluation.  Models 
exist  at  many  different  levels  of  formality,  ranging  from  verbal 
analytic  statements  of  principles  to  closed-form  mathematical 
solutions.  The  term  model  means  something  quite  different  to  a 
social  psychologist  than  to  a physicist  or  engineering  designer. 

One  can  precipitate  extensive  philosophical  arguments  by  attempting 
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to  distinguish  a model  from  a theory.  Rather  than  attempting  a 
formal  definition  of  a model,  we  have  developed  the  following  set 
of  criteria  for  deciding  whether  to  include  a particular  model  in 
our  review.  Several  of  these  criteria  are  pragmatic.  Others 
relate  to  what  we  believe  are  the  essential  characteristics  of 
quantitative  predictive  models. 

1.  Relevance . We  are  taking  as  our  primary  definition  of 
relevance  the  kinds  of  activities  required  of  operators  in  the 
RPV  command  and  control  system.  We  believe  this  system  is 
representative  of  many  of  the  characteristics  of  highly -interact ive, 
large-scale , mul ti-opera tor  systems  in  general  and  of  a variety  of 
tactical  command  and  control  systems  in  particular.  We  believe 
this  focus  is  not  overly  restrictive  of  the  more  general  domain 

of  man-machine  systems  of  interest.  Specifically,  we  were  seeking 
models  for  the  following  activities: 

a)  Target  detection  and  identification. 

b)  Supervisory  monitoring,  display  scanning,  and 
surveillance. 

c)  Target  tracking  and  manual  control. 

d)  Alphanumeric  data  entry. 

e)  Graphical  data  entry. 

f)  Mental  computation,  estimation,  and  probability 
assessment. 

g)  Algorithmic  problem  solving. 

2.  Quantitative  Predictions.  Many  models  encountered  in 
scientific  journals  provide  a structure  for  a process  or  propose 
hypotheses  about  how  an  activity  should  be  represented,  but  are 
not  carried  through  to  the  point  of  making  quantitative  predictions 
about  observable  outputs  or  performance  measures.  Such  models  arc 
of  little  value  for  our  purposes. 
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3.  Computational  Complexity.  We  arc  excluding  from  consider- 
ation models  whose  computational  complexity  is  so  great  as  to 
preclude  practical  application  to  problems  of  interest. 

4.  Parameter  Determination.  In  order  for  a model  to  be 
predictive,  its  parameters  must  be  determinable,  either  a priori 
from  other  sources  or  from  the  constraints  imposed  by  a particular 
application.  A difficulty  with  many  of  the  models  available  in 
the  literature  is  that  their  parameters,  if  specified  fit  all,  have 
been  determined  only  for  a specific  laboratory  task  and  are 
unlikely  to  generalize  to  other  contexts.  While  we  have  included 
several  models  subject  to  this  criticism,  we  have  tried  to  point 
out  their  limitations. 

5.  Reliability . A model  is  reliable,  by  definition,  if 
repeated  applications  yield  the  same  or  substantially  similar 
results.  It  should  be  noted,  however,  that  it  is  sometimes 
difficult  to  judge  the  reliability  of  recent  models,  simply  because 
many  of  them  have  not  yet  had  the  benefit  of  repeated  application. 

6.  Val ideation.  Tests  of  the  validity  of  a model  may  range 
from  informal  demonstrations  that  it  produces  reasonable  results 
to  formal  tests  of  the  goodness  of  fit  of  the  model's  predictions 
to  independent  data.  We  are  excluding  from  consideration  proposed 
models  for  which  no  attempts  at  validation  have  been  made.  Formal 
tests  of  validity,  however,  are  likely  to  exist  only  for  models, 
addressing  individual  tasks  or  components  of  human  performance. 

i For  evaluating  larger  scale  systems  models,  less  formal  evaluations 

may  be  the  most  we  can  expect. 
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Thus  our  goals  have  been  to  identify  particular  models  that: 
are  sufficiently  explicit  and  have  promise  for  appl ication  to 
military  command  and  control  system  modelling  activities.  In  the 
course  of  this  search  we  have  also  sought  to  identify  issues  and 
problems  that  limit  model  usefulness  and  that  must  be  addressed 
if  we  are  to  achieve  the  broader  goal  of  useful  man-machine 
modelling  technology. 

1 . 2 Outline  of  Report 

The  remainder  of  this  report  is  organized  into  four  main  sections. 
In  Section  2 we  have  provided  a detailed  and  critical  evaluation  of 
the  classes  of  human  performance  models  wo  have  examined.  This 
review  is  organized  on  the  basis  of  various  approaches  to 
modelling  rather  than  the  contexts  to  which  the  models  are 
applicable.  We  believe  this  will  provide  a more  coherent  presen- 
tation of  the  state-of-the-art.  This  section  concludes  with  an 
examination  of  the  interrelations  among  existing  modelling  efforts 
and  an  evaluation  of  the  gaps  between  needs  and  available  models. 

In  Section  3 we  discuss  the  issues  and  problems  that  we  have 
identified  in  our  review  of  models  and  modelling  approaches. 

j Section  4 describes  recommendations  for  further  research  that  is 

needed  to  advance  our  ability  to  use  modelling  as  a system  design 
tool.  Finally/  in  Appendix  A we  provide  abstracts  of  the  models 
we  have  found  that  we  believe  have  some  measure  of  applicability 
.in  the  system  modelling  context. 

I 
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2.0  CRITICAL  SURVEY  AND  REVIEW  OF  MODELS  POTENTIALLY  USEFUL 

FOR  SIMULATION  MODELLING 

2.1  D a*- a Bank  To  rim  i lati  on  r> 

It  became  evident  during  efforts  in  the  early  1930's  to 
incorporate  estimates  of  human  reliability  into  estimates  of  total 
system  effectiveness  that  relevant  human  performance  data  did  not 
exist  in  useable  form.  The  first  attempt  to  extract  such  data 
from  the  literature  of  general  and  applied  psychology  and  human 
factors,  and  to  represent  it  in  a tabular  array  useable  by  design 
engineers,  was  conducted  by  Payne  and  Altman  at  the  American  Institutes 
for  Research  (AIR)  in  1962  (see  Keister,  1964).  Called  "Data  Store," 
this  compilation  describes  the  characteristics  of  various  controls 
and  displays,  the  minimum  times  required  to  operate  specified 
controls,  and  the  reliabilities  with  which  the  operations  can  be 
expected  to  be  carried  out. 

Since  Data  Store,  there  have  been  repeated  attempts  to 
create  data  bases  from  which  human  performance  times  and  error 
rates  can  be  extracted  and  utilized.  A number  of  these,  though 
relatively  complete  with  respect  to  specification,  have  not  been 
implemented  (see,  for  example,  Meister  and  Mills,  1971;  Verdi  and 
Smith,  1966).  Of  those  that  have  been  implemented,  some  simply 
represent  extensions  of  the  list  of  elements  contained  in  the 
original  Data  Store  while  others  exhibit  differences  in  basic 
organizational  level  of  aggregation  - that  is,  in  the  definition 
of  what  behavioral  or  equipment  level  to  assume  for  tabulation  of 
atomic  events  - or  in  task  specificity. 

None  of  the  data  bank  formulat.ipns  to  date  are  models  in  the 
sense  that  we  have  used  that  term  elsewhere  in  this  report.  They 


Report  No.  3446 


Bolt  Beranek  and  Newman  Inc. 


2.1.1  Performance  Area  of  Concern 

Almost  all  attempts  to  create  data  banks  are  limited  to  a 
specific  area  of  concern  and,  because  of  the  time,  effort, 
and  cost  involved  in  developing  and  maintaining  them,  few  seem  to 
evolve  beyond  that  original  area.  Data  Store  was  originally 
conceived  in  support  of  work  in  display  and  control  evaluation, 
and  despite  an  interest  in  broadening  its  scope  (see,  for  example, 
Altman  1967) , the  formulation  remains  most  useful  when  employed  in 
connection  with  the  original  purpose.  The  SHERB  (Sandia  Human 
Error  Rate  Bank)  system  was  developed  in  the  context  of  an  interest 
in  human  performance  in  process-control  operations.  The  HERALD 
data  bank,  developed  by  engineers  at  Lockheed -Georgia , is  limited 
in  interest  to  the  interpretation  and  operation  of  displays  and 
controls  by  an  aircraft  pilot  and  crew.  Another  example  of  this 
specificity  is  the  formulation  by  Irwin,  Levitz,  and  Freed 
(1964)  . 

A possible  exception  to  the  generalization  that  data  banks 
tend  to  be  limited  in  their  purview  may  be  the  system  recently 
conceived  by  Blanchard  (1973)  for  the  Navy,  which  would,  when 
completed,  contain  twelve  or  more  smaller  banks  including  the 
following : 

1)  Operational  Performance  Baseline  Data 

2)  General  Human  Performance  Capability  Data 

3)  Psychophys.i  ologi  cal  Data 

4)  Personnel  Cost  Data 

5)  Human  Locomation  Data 

6)  Operational  Requirements  Information 

7)  Equipment  Reliability  Data 

8)  Micro-Behavior  Data  (eye  movements,  hand  movement , etc.) 

9)  Maintenance  Activity  Data 

10)  Hardware  Component  and  Specification  Dai  i 

11)  Shipboard  Work  Standards  and  Bar;  '1  i no  Da!  a 

12)  Human  Engineering  Standards  and  Specific  a ion:; 
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2.1.2  Data  Baso  Organization 

Two  basically  different  approaches  have  been  taken  to  the 
design  of  data  banks.  One  of  these,  exemplified  by  Data  Store, 
places  prime  emphasis  on  the  hardware  item  with  respect  to  which 
the  behavior  to  be  predicted  occurs.  In  this  organization,  a 
given  hardware  item  is  analyzed  in  terms  of  its  components  and 

a performance  reliability  estimate,  established  on  the  basis 
of  a study  of  available  literature  or  actual  experimentation, 
is  assigned  for  each  component.  A consecuence  of 
this  approach  is  that  the  values  on  which  the 
prediction  of  performance  at  a task  level  is  to  be  based  exist 
at  a very  molecular  level  in  the  data  base  and  must  be  aggregated 
according  to  some  appropriate  rule  in  order  to  provide  the 
desired  prediction.  Unfortunately,  the  nature  of  this  "appropriate 
rule"  is  frequently  in  doubt. 


An  alternative  to  the  molecular  approach  is  illustrated  by  the 
SHERB  organization.  Here,  the  basic  unit  of  analysis  is  the 
task  itself  and  the  procedural  context  in  which  that  task  occurs. 
Reliability  data  on  specific  types  of  errors  with  specific  types  of 
equipment  are  collected  under  such  broad  behavioral  task  descrip- 
tions as  "assembly,"  "inspection,"  "installation,"  etc.  This 
organization  has  at  least  two  consequences : (1)  it  helps  to 

obviate  the  aggregation  of  molecular  reliabilities,  at 
least  at  the  level  of  single  tasks;  and  (2)  experimental  and 
situational  factors  that  may  have  been  important  determiners  of 
the  data  employed  to  establish  the  reliability  values, and  that 
may  or  may  not  be  present  in  the  context  of  the  tasks  to  be 
pred icted, are  more  easily  accumulated  and  assessed. 
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It  is  impossible  to  decide  objectively  at  this  point  which 
of  these  two  organizations  is  the  better,  since  in  no  case  of  which 
we  are  aware  has  one  been  pitted  against  the  other  in  the  solution 
of  the  same  performance  prediction  problem.*  It  may  even  prove 
to  be  impossible  if  such  a confrontation  were  arranged,  unless  the 
latter  also  took  into  account  the  characteristics  and  training  of 
different  populations  of  users.  It  may  be  the  case,  for  example, 
that  the  molecular-type  organization  is  more  "natural"  and  leads 
to  more  successful  exploitation  by  the  hardware  and  systems  engi- 
neering communities  and  that  the  molar-type  is  more  "natural"  and 
exploitable  by  the  human  engineering  community.  To  the  extent  that 
this  is  the  case,  it  would  seem  appropriate  to  exploit  current 
methods  for  storing  data  in  minimally  constrained  computer  file 
structures  and  for  providing  access  by  whatever  interrogation 
conventions  are  felt  by  a given  user  to  be  most  productive. 


*There  may  be  a number  ot  reasons  for  this.  One  is  the  Pact  that, 
as  we  noted  earlier,  most  data  banks  are  compiled  for  the  purpose 
of  answering  questions  in  a specific  performance  area  and  have 
little  enough  in  common  that  a direct  comparison  would  bo 
inappi'opr iate . A second  is  that  the  typical  impetus  for 
developing  a new  formulation  is  dissatisfaction  with  an  old  one. 
From  one  point  of  view,  then,  the  decision  as  to  which  is  superior 
has  already  been  made  by  the  time  the  design  of  the  replacement 
is  complete,  and  there  may  be  little  incentive  to  conduct  a 
compar  ative  evalua  t.  ion . 
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2.1.3  Data  Base  Contents 

All  of  the  models  for  which  current  data  banks  are 
potentially  useful  are  concerned  with  predicting  the  reliability 
with  which  the  human  operator  will  perform  soma  assigned  task 
or  set  of  tasks.  Some  are  also  concerned  with  the  time  taken 
to  perform  the  task.  As  a consequence,  most  data  banks  contain 
both  reliability  and  completion  time  information,  either  in  the 
form  of  point  estimates  (expected  reliabilities,  minimum  or 
expected  times),  the  observed  distributions, or  parametric  data 
from  which  distributions  can  be  derived. 

Though  there  are  few  differences  among  designers  and  users 
regarding  the  most  useful  ways  to  represent  temporal  information, 
there  is  some  controversy  surrounding  the  nature  of  the  most 
appropriate  reliability  statistic.  As  indicated  in  the  summary, 

AIR  Data  Store  contains,  for  each  task  element  defined  in  the 
data  base,  a point  estimate  of  the  probability  that  that  task  element 
will  be  performed  correctly,  defined  as  unity  minus  the  aggregated 
error  probabilities  obtained  from  the  literature.  Further,  it 
contains,  for  each  critical  value  or  range  of  values  for  parameters 
associated  with  the  equipment  used,  a point  estimate  of  the 
probability  that  the  parameter  value  will  lead  to  successful 
performance.  In  this  formulation,  then,  reliability  is  defined 
simply  in  terms  of  probable  success  without  respect  to  time. 

In  some  applications,  the  rate  at  which  errors  are  produced 
is  of  critical  concern,  and  there  is  a desire  to  include  in  the 
data  base  some  indication  of  "failure  rate"  similar  to  that  defined 
with  respect  to  hardware  operation.  As  Swain  (1970)  points  out, 
such  data  tend  to  be  difficult  to  come  by  since  in  most  otherwise 
applicable  field  studies  (e.g. , in  t he  area  of  quality  assurance) , 
the  primary  interest  is  in  crrorsper  unit  time.  The  situation  is 
much  the  same  with  respect  to  laboratory  data,  since  most  investi- 
gators report  performance  .in  terms  of  total  numboi  of 
experiment  or  per  experimental  task. 

1 J 
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2.1.4  Predictive  Accuracy  as  a Function  of  Number  of  Parameters 

All  modeling  techniques  that  begin  by  analyzing  the  per- 
formance of  a given  task  into  discrete  activities  of  the  sort 
represented  in  Data  Store  must  include  some  rube  for  re- 
synthesizing these  molecular  activities  into  a more  molar 
entity,  possibly  the  original  task  itself  or  an  even  higher- 
level  aggregate  such  as  a total  procedure  made  up  of  n tasks. 

As  we  have  noted,  the  rule  used  most  often  is  the  simple 
product  rule,  in  which  reliabilities  associated  with  molecular 
activities  are  multiplied  together  to  achieve  an  estimate  at 
the  desired  level  of  aggregation. 

It  should  ba  clear  from  our  discussion  that  the  reliability 
(and  time)  values  that  appear  in  human  performance  data  banks 
represent  approximations  and  "best  estimates,"  and,  as  such, 
may  be  incorrect  for  any  given  application.  One  might  expect, 
then,  that  if  these  values  are  aggregated  with  the  product  rule, 
the  accuracy  of  the  resulting  estimate  would  somehow  depend  upon 
the  number  of  task  parameters  the  analyst  has  chosen  to  identify. 

The  only  systematic  study  of  this  possibility  uncovered  in 
our  review  was  made  by  Swain  (1967).  This  investigator  performed 
a Monte  Carlo  simulation  on  artificial  (but  meaningful)  tasks 
constructed  from  AIR  Data  Store.  Kach  iteration  provided  an 
estimate  of  the  probability  of  one  or  more  errors  in  per forinunce . 
The  resul ting "prediction , " arrived  at  by  the  rules  established  for 
use  of  Data  Store  (sec  Abstract  No.  I)  was  then  compared  against  the 
prediction  of  a nominal  mode L that  employed  the  average  value  of 
all  relevant  behavioral  unit  reliabilities  raised  to  a power  equal 
to  the  number  of  naramel.ors  considered: 


Q 
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where 

Q = the  probability  of  one  or  more  errors 
p = the  average  value  of  behavioral  unit  reliabilities 
n = the  total  number  of  parameters 


Swain  found  that  for  a large  number  of  parameters  (n  = 40) , 
the  nominal  rule  provided  a very  good  approximation  of  the  value 

derived  from  the  simulation,  but  that  for  smaller  numbers  of  para- 

{ 

meters  (e.g.,  n =10or20)  the  approximation  was  unsatisfactory. 

As  this  author  points  out,  an  important  implication  of  these 
findings  is  that 

"The  more  meticulous  one  is  in  the  analysis  (i.e., 
the  more  terms  one  uses  in  the  reliability  equation)  , 
the  lower  the  estimated  reliability  will  be.  It 
is  apparent,  then,  that  if  different  analysts  choose 
a different  total  number  of  dimensions  as  being 
relevant  to  tasks,  they  will  arrive  at  different 
estimates  of  the  reliability  of  a system  made  up 
of  these  tasks."  (p.  24) 


I 


Report  No.  3446 


Bolt  Bcranek  and  Newman  Inc. 


2 . 2 Five  No two r k -B a sod  To c h n i qucs 

The  interest  of  human  factors  specialists  in  the  formulation 
and  use  of  network  techniques  for  prediction  of  human  performance 
derives  from  a number  of  sources.  One  is  the  observation  that 
such  techniques  have  proved  useful  for  estimating  the  performance 
capabilities  of  systems  composed  of  functionally  interrelated 
items  of  hardware.  A second  is  the  realization  that  in  most 
complex  systems,  a large  number  of  functions  are  performed  by 
the  human  and  that  the  human  can  (and  possibly  ought  to  be) 
considered  in  the  same  terms  as  the  hardware  and  software  elements 
with  which  he  interacts.  A third  source  is  the  ease  with  which 
a network  approach,  combined  with  suitable  mathematical  operators, 
can  accommodate  a variety  of  data  input  types,  from  empirically- 
derived  estimates  of  central  tendency  (mean  or  median  performance) 
to  theoretically-assumed  distributions.  A fourth  and  very 
important  source,  in  the  context  of  this  report,  is  the  hypothesis 
that  a suitably  designed  network  might  serve  as  an  import  ant 
system-level  mechanism  for  the  aggregation  of  isolated  and 
independent  task  and  ^ub-task  models  currently  available  in  the 
psychology,  human  factors,  control  theory,  and  other  human 
performance  lit e natures. 


The  goal  of  almost  all  network  techniques  is  to  g -l  from  a 
statement  of  the  functional  relations  among  individual  ta.sk 
elements  to  an  estimate  of  the  performanc  of  soirn * del  ined 
aggregate  of  those  elements.  Tin?  performance  paramot-  rs  or 
interest  are  typically  one  or  more  of  the  following:  (1)  the 

tine  required  to  complete  the  t ask  aggregate;  (•')  t h<  prohibit  i I y 
that  the  task  aggregate  will  be  completed  in  a given  t ime;  ( <) 
the  probability  I lia  t the  ngnrogni"  wilt  be  co  i :i:  -.  ■ r 1 1 y- 
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that  is,  at  or  above  some  level  defined  as  acceptable.  Recause 
the  techniques  employ  concepts  similar  to  those  of  standard  reli- 
ability theory,  one  can,  if  interested,  utilize  other  performance 
measures  such  as  mean-time-to-fa i lure  (MTTF) , moan-t ime-between- 
failures  (MTBF) , etc. 

The  various  techniques,  thouyh  frequently  quite  different 
when  viewed  with  respect  to  operating  assumptions,  degree  of 
detail,  etc.,  share  common  modelling  requirements.  All  require  a 
thorough  description  (frequently  rendered  in  diagrammatic  form) 
of  actual  or  intended  tasks  and  of  the  interrelationships  among 
those  tasks.  Such  procedural  flows  or  operational  sequence 
diagrams  (OSDs)  may  sometimes  be?  supplemented  with  other  analytic 
devices  such  as  the  fault  tree,  which  provide  descriptions  of  the 
alternative  ways  a task  or  task-aggregate  could  fail  to  be 
accomplished  in  the  desired  time  or  at  the  desired  level  of 
reliability . 

A second  requirement  is  for  "data,"  either  in  the  form  of 
measures  of  central  tendency  (the  mean  or  median)  or  in  the*  form 
of  distributions,  which  can  be  taken  as  measures  of  performance  in 
each  of  the  tasks  included  in  the  aggregate.  In  some  applications, 
empirically-derived  measures  or  distributions  have  been  employed, 
whereas  in  others,  rational  estimates  or  distributions  thought  to 
represent  reasonable  approximations  have  been  utilized  where 
empirical  data  Lire  lacking. 

As  with  data  banks,  no  systematic  t axonomy  has  been  developed 
for  defining  or  segmenting  task  sequences.  What  may  be  a tusk 
process  with  associated  "data"  at  one  level  may  be  a small  notwoi  V. 
of  processes  nt  another  level,  each  with  its  own  data  which,  in 
the  aggregate,  represent  the  task. 
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A third  common  requirement  is  for  a set  of  rules  and  pro- 
cedures for  combining  estimates  of  central  tendency  or  distributions 
attributed  to  tasks  into  overall  estimates  or  distributions  repre- 
senting the  task  aggregate.  Frequently,  the  rules  and  procedures 
employed  are  those  of  standard  reliability  theory  where,  for  example, 
the  estimated  reliabilities  of  separate  components  are  multiplied 
together  to  yield  an  estimate  of  overall  reliability,  and  the  time 
required  for  each  component  to  accomplish  its  function  is  lidded  to 
that  required  by  each  other  component  in  order  to  estimate  tota]  time 
to  completion. 


At  least  two  problems  have  arisen  in  connection  with  this 
approach  to  human  operator  modelling.  One  results  from  a lack  of 
empirical  data  relating  to  expected  performance.  In  some  cases 
the  data  simply  do  not  exist.  In  other  cases,  the  validity  and/ 
or  reliability  of  available  data  are  suspect.  In  either  case, 
assumptions  concerning  expected  values  or  likely  distributions 
must  be  made,  and  these  add  an  additional  source  oT  uncertainty 
to  the  modelling  effort. 


A second  problem  concerns  the  rules  and  procedures  for 
combining  point  estimates  or  distributions.  Where  the  aggregate 
contains  some  tasks  that  are  highly  cognilire  in  content  - and 


particularly  where  a number  of  I 
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Another  general  obs*rv  • . Lon  one  might  make  with  respect  to 

the  network  approach  is  that,  while  it  may  be  suited  to  the 
modelling  of  discrete  activities,  it  historically  has  not  been 
suitable  for  modelling  of  continuous  activities.  Two  points  must 
be  made  in  this  regard,  however:  (1)  Most  networks  were  not 

developed  with  continuous  tasks  in  mind;  thus,  feti  lures  of  gen- 
eralization are  not  appropriate  criticism  of  the  approach  per  sc. 

(2)  At  least  one  network-based  approach  - SAINT  - has  been 
successfully  extended  to  include  certain  families  of  continuous 
variables;  thus,  there  seems  to  be  no  inherent  limitation  to 
continuous  task  modelling  via  network  formulations. 

We  take  the  position  in  this  report  chat  ne reworks  are  not, 
in  and  of  themselves,  models  of  human  performance,  though  they  may 
clearly  be  representations  of  system  structures  in  which  that 
performance  is  imbedded.  As  a result,  we  have  included  no  abstracts 
of  these  techniques  in  the  Appendix  of  this  report.  It  must  be 
recognized  that  this  omission  is  not  intended  as  a negative  comment 
on  their  perceived  value  or  utility  in  the  assessment  of  system 
Performance,  which,  in  our  judgment,  is  quite  high.  The  omission 
simply  reflects  the  feeling  that  the  various  network  techniques 
can  be  more  usefully  conceived  as  formal  sets  of  rules  and  pro- 
cedures for  defining  system  structure  and  that  these  structures 
then  provide  the  mechanism  for  aggregating  data  available  from  data 
banks  or  incorporating  functional  performance  models. 

T n the  remainder  of  this  section  we  discuss  four  n d.wc  rk 
techniques  that  have  been  applied  successfully  to  the  {'valuation 
of  human  performance  in  man/r.vich i no  systems.  To  thi  :■  set  w •>  have 
added  one  other  technique  --  PERT  --  which,  while  not  a structure 
for  evaluating  human  performance  per  so,  provide.;  the  f j am. 'work 
from  which  several  current  techniques  are  cieriv  -d , and  which 
serves  as  a convenient  reference  for  di :u;:o  i on  . 
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The  set  of  four  human  performance  network  techniques  selected 
illustrates  quite  well  the  variety  of  approaches  currently 
available  for  prediction  of  performance.  Further,  the  chosen 
techniques  demonstrate  what  we  consider  to  be  the  streuqths  and 
weaknesses  of  network  methods.  It  is  important  to  realize  that 
we  have  not,  by  any  means,  exhausted  the  useful  techniques  in 
this  class.  For  reviews  of  other  techniques  of  the  types 
illustrated  here,  the  reader  is  encouraged  to  read  ‘leister' s 
(1971)  comparative  analysis  of  reliability  models. 
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2-2.1  PERT 

The  application  of  PERT  follows  essentially  the  formula  laid 
down  above.  One  begins  by  constructing  a network  consisting  of 
all  major  activ  ties  intervening  between  the  start  and  the  completion 
of  a task.  (Miller,  1963)  Particular  pains  are  taken  to  portray 
accurately  all  sequential  dependencies  implied  in  the  task  and  to  includ 
activities  that  can  or  are  intended  to  be  conducted  in  parallel. 

Following  completion  of  the  network,  three  estimates  are 
made  concerning  each  activity:  (1)  the  earliest  possible  and 

(2)  latest  possible  or  allowable  times  at  which  it  could  be 
completed;  and  (3)  the  most  likely  time  at  which  the  activity 
would  be  completed.  It  is  then  assumed  that  these  estimates 
represent  the  end  points  and  mode,  respectively,  of  a Beta 
probability  density  function  with  mean,  t , equal  to 


t + t -l-  4tr 
op  f 


where  t = 

o 

the 

earliest  tine 

t 

P 

the 

latest  tine 
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mode 
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a variance. 
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With  expected  times  and  variances  thus  derived  for  each 
activity , one  then  estimates  the  total  task  time,  T , as  the  sum. 
of  the  individual  expected  iictivity  times, 


n 


2 

and  estimates  the  variance  in  that  time,  a (T  ) , as  the  sum  of 
the  individual  activity  variances. 


One  can  also  estimate  the  probability  that  the  task  will 
be  completed  as  of  a specified  time  - frequently,  a desired 
project  end-date.  To  accomplish  this,  one  assumes  that  the 
composite  of  activity  completion  times  has  a normal  distribution 

and  then  computes  the  ratio  of  the  area  under  that  distribution 
to  the  left  of  the  specified  time  to  the  area  under  the  total 
distribution.  In  units  of  standard  deviation  (a) , this  becomes 


where  T^  = the  specified  tine  and  T{i  is  defined  as  above. 


There  are,  of  course,  many  other  facets  to  a Pd RT  analysis. 
Ordinarily  one  is  interested  in  such  things  as  the  critical  (that 
is,  the  longest)  path  through  the  network,  the  distributions  of 
positive  and  negative  "slack"  among  activities,  etc.,  for  purposes 
of  controlling  a total  effort.  What  is  of  major  interest  here, 
however , are  the  strengths  and  weaknesses  of  that  portion  of  the 
model  sketched  above. 


I 

I 
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This  technique  has  two  primary  strengths.  First,  there  can 
be  little  doubt  that  the  development  of  a network  in  detail 
sufficient  for  the  application  of  PERT  forces  one  to  think 
critically  about  the  relationships  among  activities.  In  specifying 
activities  at  a fine  level,  the  analyst  can  often  determine  in  some 
detail  what  he  does  not  know  and  what  information  he  must  acquire 
to  assess  the  task. 

Secondly,  the  technique  provides  quantitative  information 
useful  for  the  assessment  of  project  completion  time  and  provides 
a framework  for  assessing  performance  reliability.  With  this 
information  and  framework,  one  can  hope  to  characterize  the 
responsiveness  of  an  intended  mission  and  to  consider  alternative 
configurations . 

The  weaknesses  we  see  have  mainly  to  do  with  the  means  by 
which  the  temporal  data  characterizing  an  activity  are  acquired, 
the  distributions  assumed,  and  the  mathematical  simplifications 
employed.  As  indicated,  the  analysis  requires  estimates  of 
earliest,  latest,  and  modal  times.  These  frequently  represent 
either  the  opinions  of  individuals  familiar  with  activities 
similar  to  those  under  study  or  the  outputs  of  part-task  studies 
or  laboratory  experiments. 

There  seems  every  reason  to  expect  that  the  pattern  of 
interaction  that  gives  rise  to  the  expert's  estivates  or  the 
experimental  results  will  be  different  from  those  that  will 
operate  in  the  task  being  assessed,  unless  one  is  so  familiar 
with  both  contexts  as  to  be  able?  to  ensure  the  validity  of  the 
estimates  being  used  for  analysis. 

I 


i 


Report  No.  34 4 6 


Bolt  Beranek  and  Newman  Inc. 


In  addition,  much  concern  has  been  shown  in  the  literature 
over  the  assumption  that  the  time  estimations  employed  in  PERT 
are  fitted  into  a Beta  distribution  with  a variance  equal  to  one- 
sixth  of  the  range  between  the  earliest  and  latest  times.  Three 
possible  sources  of  error  arising  from  this  assumption  have  been 
studied  empirically  by  MacCrimmon  and  Ryavec  (1964),  who  found 
that  errors  of  30  percent  and  15  percent  can  be  introduced  into 
the  calculation  of  the  expected  time  and  standard  deviation  as  a 
result  of  failures  to  meet  this  assumption.  They  also 
found  that  approximately  the  same  sorts  of  error  rates  would  have 
been  observed  with  a much  simpler  triangular  distribution.  Thus, 
although  the  Beta-distribution  has  strong  intuitive  appeal,  it  may  . 
be  open  to  question  on  empirical  grounds. 

The  assumption  concerning  normality  has  more  in  its  favor, 
since  it  is  based  on  the  Central- Limit  Theorem.  The  critical 
question  in  any  particular  application,  of  course,  is  whether  or 
not  there  is  a sufficient  number  of  activities  in  a sequence  to 
justify  the  assumption  that  the  distribution  of  sample  means  is 
approximately  normal.  Meister  (1964)  mentions  in  this  regard 
an  unpublished  study  by  Rook,  who  found  that  a distribution  of 
failure  probabilities  may  be  considered  normal  if  the  number  of 
parameters  exceeds  15. 
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Finally,  there  is  the  question  of  whether  or  not  the  mean 
and  variance  of  the  total  task  time  are  equal  to  the  sums  of  the 
individual  means  and  of  the  individual  variances,  respectively. 
Research  on  this  point  has  suggested  that  the  answer  depends 
critically  on  the  relative  lengths  of  paths  in  the  network  and 
the  number  of  activities  shared  in  common  by  different  paths. 
This  is  of  great  concern,  it  seems  to  us,  for  it  suggests  that 
even  when  one  can  be  confident  of  his  temporal  data  and  of  its 
distribution,  the  very  nature  of  the  task  structure  may  preclude 
a precise  analysis. 

It  should  be  apparent  that  the  MacCrimmon  and  Ryavec 
analysis  is  related  only  to  temporal  aspects  of  the  mode] s 
assumptions  and  not  at  all  to  the  concept  of  reliability 
therein.  Since,  however,  the  accuracy  of  the  reliability 
assessment  is  based  entirely  on  estimates  of  total  allowable 
task  time  and  on  the  derivation  of  expected  time,  errors  in 
computed  reliability  can  be  expected  to  be  monotonically 
related  to  errors  in  total  expected  project  time. 
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2.2.2  The  Siegel  and  Wolf  Technique 

The  Sioael  and  Wolf  Model  (SWM),  which  appeared  in  1961, 
has  been  of  continuing  interest  to  human  performance  analysts 
and  recently  has  been  extended  to  the  study  of  survivability/ 
vulnerability  in  nuclear  environments  (Chubb,  1971).  It 
embodies  some  of  the  network  concepts  of  PERT  and,  like  that 
earlier  formulation,  depends  significantly  on  relationships 
between  required  task  times  and  available  task  times  for  measures 
of  completion  probability  and  stress.  Unlike  PERT,  however,  it 
is  a humam  performance  prediction  technique  and,  even  more  signi- 
ficantly, a technique  employing  Monte  Carlo  simulation . It  is  also 
built  upon  conceptual  structures  and  empirical  observations  drawn 
from  psychology  and  human  factors  and,  as  such,  can  be  considered 
a model  in  its  own  right. 


An  indication  of  the  analytic  interests  represented  by  SWM 
is  provided  by  the  following  quote: 

"It  is  the  purpose  of  this  model  to  give  the  equipment 
designers  quantitative  answers  to  questions  such  as  the 
following  while  the  equipment  is  in  the  early  design 
s tage : 


(1)  Can  an  average  operator  be  expected  to  complete 
successfully  all  actions  required  in  the  performance 
of  a task  within  a time  limit,  T,  for  a given  opera- 
tor procedure  and  a given  machine  design? 

(2)  How  does  success  probability  change  for  slower  or 
faster  operators  and  longer  or  shorter  periods  of 
allotted  time? 

(3)  How  great  a stress  is  placed  on  the  operator 
during  his  performance  and  in  which  portions  of  the 
task  is  ho  overloaded  or  underloaded? 

(4)  What  is  the  frequency  distribution  of  operator 
failures  as  a fund  ion  of  vai  ions  s ' ress  tolerances 
and  operator  speeds?"  (Siegel  & Wolf,  1961). 
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To  apply  the  model,  one  first  performs  an  analysis  and 
identifies  which  tasks  are  essential  to  completion  of  the  mission 
and  which  are  unessential.  For  each  of  these,  the  following  data 
are  specified,  using  "the  best  available  sources": 

(1)  The  average  time  required  by  the  operator  to 
perform  each  task. 

(2)  The  average  standard  deviation  about  the  average 
time . 

(3)  The  average  probability  of  successfully  per- 
forming each  task. 

(4)  An  estimate  of  the  extent  to  which  successful 
performance  of  each  task  is  required  for  completion 
of  the  total  mission. 

(5)  The  waiting  time  (if  any) . This  represents  the 
time  elapsed  between  the  start  of  the  mission  and 
the  st^art  of  each  task,  during  which  no  action  can 
be  taken  by  the  operator. 

(6)  The  next  task  to  be  performed  given  failure  to 
accomplish  a current  task. 

(7)  The  next  task  to  be  performed  given  successful 
accomplishment  of  a current  task. 

Following  the  determination  of  these  values,  one  then 
computes  urgency  and  stress  condi  1 ions,  which  are  defined  with 
respect  to  the  available  tin<-  and  sequential  constraints  of  the 
netw  >rk . (See  Al >s tract  Nos . ' and  > . ) A situ  is  " non-ur gent* 
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"non-urgent"  or  "urgent,"  one  assigns  a stress  level  with  a value 
of  unity.  When  it  is  "highly  urgent"  the  level  is  defined  as 
the  ratio  of  the  sum  of  the  average  time  required  to  complete 
remaining  required  tasks  to  the  total  available  time  remaining 
in  the  mission* 


j.  a.  . 

1 

where  s.  = the  stress  level 

l 

T? 

TV  = the  average  time  required  to  complete  all  remaining 
required  tasks  in  the  mission 

T = the  total  time  available  for  completing  all 
tasks  in  the  mission 

tV  = the  total  time  required  to  complete  all  tasks  through 

(i  - 1) 

An  important  part  of  SW'M  is  its  incorporation  of  the 
psychological  concept  of  a "stress  threshold."  As  the  authors 
note,  "Theory  suggests  that  emotion  or  stress  up  to  a certain 
point  acts  as  an  organizing  agent  on  behavior;  beyond  this 
point  stress  acts  as  a disorganizing  agent"  (1961;  p.  20). 

The  model  simulates  this  effect  by  comparing  the  value  of  s. 
against  a value  M,  input  by  the  user  prior  to  each  simulation 
run,  as  explained  below. 


* ; o r pu rpos<  *s 
shall  follow 
contained  in 


of  simplifying  this  presentation,  where  possible,  wo 
the  SLoqel  & Wolf  (1961)  not  at.  ion , rather  than  that 
later  publications. 
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With  the  stress  levels  computed,  one  then  computes  the 
time  required  for  completion  of  the  subtask,  based  on  the 
assumption  of  a normal  distribution  with  mean  and  standard 
deviation  equal  to  those  specified  earlier.  A specific  value 
for  a given  "exercise"  of  the  model  is  chosen  via  a Monte  Carlo 
technique  that  samples  from  the  distribution  of  values  assigned 
to  each  subtask.  The  computational  procedure  utilized  generates 
a sample  value  in  which  the  average  completion  times  and  average 
deviations : 

"(1)  are  used  unchanged  when  stress  equals  unity; 

(2)  are  decreased  with  increasing  stress  until  stress 
assumes  the  threshold  value; 

(3)  are  used  unchanged  when  stress  equals  the  threshold 
va 1 ue ; 

(4)  are  increased  linearly  with  increasing  stress  beyond 
M until,  when  stress  equals  M+l,  the  contributions  of 
t^  (average  completion  time)  and  oh  (average  standard 
deviation)  remain  constant  at  2t^  and  3en  , respectively." 
(Siegel  & Wolf,  1961).  Note:  t ^ and  a.  are  the  average 
time  to  complete  task  i and  the  average  standard 
deviation  of  the  average  time,  respectively . 

Finally,  one  determines  whether  or  not  the  task  can  be 
considered  to  have  been  completed  successfully.  Critical  to  the 
computation  here  are  the  nominal  probability  of  succ  as  specified 
earlier,  the  stress  level , and  the  stress  threshold.  The  co.  gutn- 
t iona 1 nil  a i s as  foil ows : 
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(l-pi)(si-l) 

p.  + --—i for  s.  < M (sea  footnote) 

M — 1 1 

Pf ( Sf+l-M) + (M-Sf ) for  M < si  < M+2 

2p.  - 1 for  s.  > M+l 

L l l 

where  p_.  = probability  of  successful  performance  of  task  i 

p^  = average  probability  of  successful  performance  of 
task  i 

s^  = stress  level  of  task  i 
M = stress  threshold 

As  Siegel  and  Wolf  note,  the  probability  of  success  thus  de- 
fined increases  in  linear  fashion  with  stress, reaching  unity  at  the 
stress  threshold.  At  this  value,  it  assumes  the  average  (i.e., 
input)  value,  p^ , and  then  decreases  linearly  until  it  reaches 
a constant  value. 

Since  the  outcome  must  be  considered  to  be  a binary  event, 
a decision  as  to  whether  the  task  was,  in  fact,  completed  success- 
fully requires  the  comparison  of  p.  with  a pseudo-randomly 
generated  number,  R.  The  decision  rule  specifies  successful 
performance  if  p.  ■ R;  otherwise,  performance  is  considered 
unsuccess f ul . 

*An  error  in  the  specification  of  the  multiplier  is  coni  iiivd  in 
the  1 969  version  of  this  argument.  The  correct  form,  anal  > j>uis 
to  that  given  here,  i. s 


( 1 -p . ) ( s . . - 1 ) 

ii  ii 
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In  actual  use,  the  computations  alluded  to  above  are 
performed  by  computer , which  also  keeps  track  of  the  total  time 
remaining  for  performance  of  the  mission.  When  that  time  has 
bean  exhausted  or  when  the  task  has  been  completed  successfully, 
the  simulation  is  terminated.  At  the  conclusion  of  a run,  data 
pertinent  to  task  completion  times  and  reliabilities  are  recovered 
for  analysis. 


An  important  aspect  of  SWM  is  its  suitability  for  use  in 
simulations  of  the  performance  of  multi-person  systems.  In  this 
role,  the  technique  employs  a number  of  self-contained  models  to 
determine  expected  task  time  as  a function  of  group  performance 
proficiency,  overtime  load,  morale,  number  of  persons,  and 
nominal  (average)  time. 

Some  feeling  for  the  amount  and  specificity  of  data 
required  as  inputs  to  the  Siegel  and  Wolf  group  simulation  is 
provided  by  the  authors’  listing  of  "mission  data"  that  must 
accompany  the  definition  of  each  action  unit  (task)  in  a total 
crew  assignment: 

(1)  The  average  time  h ^ in  tenths  of  hours,  required  to 
complete  action  unit  u on  day  d. 

(2)  An  indication  of  whether  or  not  h , represents  a 

ud  1 

fixed  Lime  period,  a variable  time  period,  or  a 
time-limited  period.  A time-limited  period  is  one 
that  must  end  by  a specified  time  of  day  regardless 
of  the  starting  time  (variable  0,  fixed  -■  1, 

1 i mi  ted  = 2 ) . 

(3)  An  indication  of  whether  or  not:  the  action  unit 
performance  is  to  be  repeated  or  improved  in  the  event 
that  performance  eff  iciency  doe:;  net  come  up  to  expecta- 
tion (no  rnp'l  it  ion  0,  repetition  or  touch  up  1). 


1 
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(4)  An  indication  of  whether  or  not  conununication  between 
stations  is  required  on  this  action  unit  (yes  = 1,  no  - 0). 

(5)  The  carry-over  code  that  specifies  the  importance  of  the 
action  unit: 

essential  = 1 

postpone  to  avoid  overtime  = 2 

postpone  if  crew  morale  is  _ 
below  the  threshold 

ignore  to  avoid  overtime  = 4 

(6)  The  orientation  values  of  the  action  unit: 

benefit  to  individual,  S 

ud 

benefit  to  crew,  C , 

ud 

benefit  to  mission,  M , 

ud 

(7)  The  type  of  action  unit--an  indicator  of  the  preference  in 
selecting  group  members: 

normal,  prefer  primary  specialty  --  1 
training,  prefer  alternate  specialty  = 2 
difficult,  insist  on  primary  specialty  = 3 

(8)  A prior  action  unit  number  q j tluit  must  be  completed 
before  action  unit  u can  be  initiated. 

(9)  Indicators  e ^ to  specify  for  each  equipment  system  whether 
or  not  it  is  required  to  accomplish  the  action  unit  (yes  - 1, 
no  - 0 for  each  equipment,  c).  (Siegel  & Wolf,  1969,  pp.  7.1-72). 

In  ad  lit  ion  t o thes  ■,  the  model  makes  provisions , tifhere 
'relevant,  for  use  of  a variety  of  input  data  relating  to  equipment 
used  (failure  rates,  repair  times,  etc. ) , to  personnel  paramet  rs 
(trai  ning  and  operational  experience!,  prof  ioiency , et  e. ) , t o 
crew  eoh  • i vein  ■ :s  and  mil  tie,  et  c. 


W. 
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The  ability  of  the  group  simulation  model  to  predict  empirically 
and  independently  verifiable  outside  criterion  data  has  been  tested 
in  the  context  of  a realistic  nuclear  missile  submarine  training 
mission  (Siegel  and  Wolf  1969).  This  check  on  the  validity  of  the 
model  had  a number  of  specific  purposes:  (1)  to  compare  the  model 

predictions  of  required  crew  composition  with  results  obtained 
from  field  experience;  (2)  to  compare  model  predictions  of  average 
crew  proficiency,  supervisor  satisfaction,  and  number  of  unessential 
work  units  remaining  unperformed  with  estimates  of  those  parameters 
by  operating  personnel;  and  (3)  to  assess  the  internal  consistency 
of  results  relating  crew  efficiency,  productivity,  cohesiveness, 
and  morale  as  a function  of  crew  size.  The  predictions  made  by  the 
model  appeared  to  accord  well  with  the  expectations  and  predictions 
of  FBM  personnel,  indicating,  from  the  authors'  point  of  view,  a 
generally  valid  model  development. 

In  addition  to  the  benefits  derived  from  analyzing  operating 
procedures  in  sufficient  detail  to  develop  the  network  representa- 
tion, SWM  has  a number  of  significant  virtues.  One  is  a mechanism 
for  modifying  performance  in  accord  with  stress.  A second  is  the: 
Monte  Carlo  component  of  the  technique.  Rather  than  predicting  a 
judgment  of  the  performance  of  the  system  on  the  sum  of  single 
subtask  expected  values,  as  in  PERT,  SWM  expressly  considers  that 
operators  will  differ  in  their  performance  of  the  same  subtask  and 
that  the  same  operator  will  exhibit  in  successive 

repetitions.  This  capacity  to  encompass  both  with j n-operator  and 
be tween-opera tor  variability  is,  in  our  judgment,  an  important 
desideratum  in  performance  modelling.  A third  is  the  ability  to 
address  multi-person  systems  problems. 
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Weaknesses  that  v/c  see  are  similar  in  some  respects  to  those 
encountered  in  PERT.  There  is,  first  of  all,  the  question  of 
whether  valid  structural  representations  and  data  can  be  assembled 
for  a system  in  which  possible  interactions  nay  be  only  dimly 
understood.  Secondly,  there  is  a question  concerning  the  soundness 
of  the  assumption  that  the  means  and  standard  deviations  specified 
are,  in  fact,  the  means  and  standard  deviations  of  normal  distri- 
butions. Though  such  an  assumption  might  be  justified,  as  in  PERT, 
in  terras  of  the  Central  Limit  Theorem,  results  of  a very  recent 
study  by  Mills  and  Hatfield  (1974)  suggest  that  the  three  parameter 
Weibull  (or  weighted  Weibull)  distribution  may  provide  a better  fit 

Thirdly,  we  have  some  concern  over  the  ability  of  SWM  — or, 
indeed,  any  network  technique  — to  yield  accurate  predictions  of 
performance  in  situations  where  observable  task  density  is  low 
(as,  for  example,  in  tasks  requiring  a groat  deal  of  monitoring 
and  signal  detection  but  only  occasional  system  input)  or  where 
the  performance  of  several  operators  functioning  in  parallel  must 
be  assessed.  The  first  of  these  situations  is  of  concern  not  only 
because  the  network  itself  is  difficult,  to  construct  when  activitie 
arc  largely  implicit,  but  also  because  sufficient  characterization 
of  those  activities  for  purposes  of  assembling  parametric  input  dal 
may  not  be  achievable.  Our  concern  in  the  latter  situation  is 
that  the  reliability  of  the  estimate  may,  as  in  PERT,  depend  on  the 
network  configuration  and  the  d agree  of  interconnectedness  of  the 
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2.2.3  SAINT 

Unlike  PERT,  SAINT  (Systems  Analysis  of  Integrated  Networks 
of  Tasks)  is  a network  technique  that  embodies  a variety  of  types 
of  process  branching,  a sot  of  alternative  distributions  for  use 
in  modelling  individual  task  elements,  and  a Monte  Carlo  procedure 
for  sampling  from  the  distributions  specified.  Designed  to  capture 
the  best  features  of  the  Siegel  and  Wolf  approach  and  the  networking 
and  symbol  manipulation  capabilities  of  GERT  (Graphical  Evaluation 
and  Review  Technique;  see  Pritsker  and  Happ,  1967) , SAINT  contains 
a Fortran-based  user  language  and  a formal  symbolism  within  which 
to  represent  and  code  critical  parameters  at  each  network  node. 

SAINT  has  been  employed  in  a wider  range  of  system  modelling 
contexts  than  any  other  network  approach  identified  in  this  review. 
The  following  are  illustrative: 

1)  Choice  reaction  time  (Harm  and  Kuperman , 1976) 

2)  Remotely  piloted  vehicle/drone  (RPV)  control  facility 
(Wortman,  Duke t,  and  Seifert,  1975) 

3)  Digital  Avionics  Information  System  (DAIS)  design 
(Kuperman  and  Seifert,  1975) 

4)  Hot  strip  mill  process  control  (Maltus  and  Buck,  1975) 

5)  Airborne  warning  and  control  systems  (Mills,  3976) 

In  this  section  we  shal I attempt  to  highlight  those  charac- 
teristics of  the  technique  that  appear  to  play  major  roles  in  its 
utility  and  generality.  Readers  interested  in  specific  applica- 
tions and  in  the  design  philosophy  and  internal  organization  of 
SAINT  are  urged  to  read  the  above  references  and  the  system 
documents  provided  by  Pritsker  et  al  ( 1974)  and  Wortman,  et  a 1 ( 1974), 
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2 . 2 . 3 . 1 Task  Concepts 

As  in  PERT  and  SWM , tasks  are  related  to  each  other  in  SAINT 
by  precedence  relationships.  These  precedence  relationships 
specify  the  flow  of  operations  through  the  network  and  indicate 
which  tasks  can  be  started  as  a result  of  the  completion  of  given 
prior  tasks.  Unlike  PERT,  the  completion  of  particular  tusks  in  a 
SAINT  network  can  modify  precedence  relations  arom.g  later  tasks, 
thus  altering  the  flow  through  the  network. 

A given  task  has  associated  with  it  input , task,  and  output 
parameters  that  specify  the  number  and  nature  of  predecessor 
tasks  that  must  bo  completed  before  the  task  in  quasi i on  can 
be  begun,  characteristics  of  the  task  and  statistical  information 
to  be  collected  and  the  branching  (to  other  tasks)  to  be  performed 
at  its  conclusion,  respectively.  A brief  summary  of  important 
parameters,  in  addition  to  those  that  specify  flow  through  the 
network,  is  presented  below  for  the  purpose  of  highlighting 
the  modelling  flexibility  that  is  achievable  through  X he  SAINT 
technology : 

2 . 2 . 3 . 2 Task  Dura t ion 

The  time  to  perforin  a task  is  specified  in  terms  of  a desired 
sampling  distribution  and  a parameter  set  id  arified  by  the  user. 
The  following  distributions  are  available: 


1) 

Constant 

7) 

Beta 

2) 

Norma  1 

8) 

Gamma 

3) 

Uni l orm  (Rectangu 1 ir) 

9) 

Beta  fitted 

! to  t hi 

4) 

Hr  1 any 

1 0) 

Scaled  coni; 

;t  ant 

5) 

Lognorma 1 

1 1 ) 

Tr  i ar  ral  ai 

6)  Po.i  ss  >n 
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If  none  of  the  above  functions  are  appropriate  to  a given 
task,  but  an  alternative  can  bo  specified,  the  SAINT  language 
provides  a means  by  which  the  user  can  write  the  alternative 
function  as  a subroutine  and  employ  it  in  the  simulation. 

2 . 2 . 3 . 3 Task  Essentiality 

SAINT  provides  a mechanism  whereby  tasks  may  be  graded  in 
terms  of  their  essentiality  to  the  total  system  missior . When, 
during  simulation  of  the  mission,  remaining  time  grows  critica  ly 
short,  less  important  tasks  may  be  skipped  in  order  to  assure  the 
completion  of  essential  tasks. 

2. 2. 3. 4 Task  Type 

Six  possible  task  types  can  be  defined  in  SAINT: 

1)  Single  Operator  Task 

2)  Joint  Operator  Task 

3)  One  of  several  operators  (first  to  become  available 
completes  job) 

4)  Hardware  Task  (no  human  operator) 

5)  Cyclic  Task 

6)  Gap  Filler  Task  (completed  only  if  time  available) 

2 . 2 . 3 . 5 Task' Output 

After  a task  has  been  completed,  a decision  is  made  as  to 
which  of  n tasks  should  next  be  started.  Since  the  set  of  possible 
next  tasks  is  defined  by  the  network,  this  decision  results  in 
the  selection  of  m from  among  the  n possible  branches  connecting 
the  completed  task  with  remaining  tasks.  Five  decision  rules 
are  implemented  in  SAINT: 


I 
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1)  "Deterministic,"  in  which  all  n branches  are  selected. 

2)  "Probabilistic,"  in  which  one  branch  is  selected  on 
the  basis  of  a random  number  drawn  from  a uniform 
distribution. 

3)  "Conditional,  'take  first',"  in  which  the  first  branch 
staisfying  a specified  condition  is  selected.  Possible 
conditions  are  (a)  associated  task  complete;  (b)  accrued 
simulation  time  equal  to  or  less  than  a specified  time; 
(c)  associated  task  not  completed;  (d)  accrued 
simulation  time  greater  than  a specified  time. 

4)  "Conditional,  'take  all',"  similar  to  the  preceding  rule 
except  that  all  m branches  satisfying  stated  condition 
are  taken. 

5)  "Modified  probabilistic,"  similar  to  "probabilistic," 
except  that  briinch  probabilities  are  qualified  by  the 
number  of  previous  completions  of  the  task  from  which 
the  branches  emanate. 


2 . 2 . 3 . 6 Statistics  Collected  on  Tasks 

Five  types  of  statistics  relative  to  temporal  aspects  of 
performance  can  be  collected  and  tabulated  in  SAINT. 

1)  Time  of  first  completion  of  a given  task 

2)  Time  of  all  completions  of  a given  task 

3)  Time  between  (successive)  completions  of  a given  task 

4)  Time  required  to  complete  a given  task 

5)  Time  elapsed  from  completion  of  ih  first  predecessor 
task  to  the  start  of  a given  task. 
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In  addition  to  those  parameters  that  characterize  task 
structures  in  SAINT,  there  is  a set  that  characterizes  the  opera- 
tor (s)  and  that  serves  "to  make  a mission  operator-specific" 
(Pritsker,  et  al,  1974;  p.  17).  The  essential  members  of  this 
set  are  summarized  briefly  below: 

2 . 2 . 3 . 7 Operator  Speed 

Values  assigned  to  this  parameter  reflect  differences  in 
task  performance  time  caused  by  differences  in  proficiency  and 
training  among  individual  operators  and  groups  of  operators. 

2 . 2 . 3 . 8 Operator  Accuracy 

Values  assigned  to  this  parameter  reflect  differences  in 
performance  accuracy  due  to  differences  in  proficiency  and  training 
among  individual  operators  and  groups  of  operators. 

2 . 2 . 3 . 9 Operator  Stress  Due  to  Workload 

As  in  SWM,  the  stress  imposed  on  an  operator  is  defined  as 
the  ratio  between  the  time  required  to  complete  remaining  tasks 
and  the  time  available  to  complete  them.  In  addition,  the  basic 
SWM  principles  associated  with  performance  accuracy  and  time  as 
a function  of  stress  level  are  contained  in* SAINT. 

2.2.3.10  Goa Grad.i  en  t 

The  commonly  observed  increase  in  the  nominal  accuracy  of 
performance  as  one  nears  completion  of  a task  can  bo  simulated. 

To  approximate  this  effect,  the  SAINT  user  prescribes,  for  each 
modelled  operator,  a value  reflecting  the  percentage  of  mission 
completion  required  before  onset  of  the  gradient.  The  gradient 
itself  is  produced  by  adding  to  the  nominal  probability  of  success 
a value  whose  size  is  a direct  function  of  the  extent  of 
prescribed  task  completion. 
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2.2.3.11  Comment 

As  a simulation  utility  that  employs  a bottom-up  approach 
to  performance  prediction,  SAINT  is  probably  without  peer  at  this 
time.  It  incorporates  what  we  consider  to  bo  the  most  satisfactory 
concepts  with  respect  to  task  and  operator  parameters  identified 
in  SWM , and  employs  a high  level  language  that  is  easily  learned 
and  manipulated  by  the  user.  Further,  the  very  flexible  branching 
structure  and  the  capability  for  changing  the  sequence  of 
subsequent  tasks  offer  what  is  perhaps  a unique  opportunity  for 
the  simulation  of  system  missions  with  broad  dynamic  range. 

Like  other  network  approaches,  however,  the  validity  of 
SAINT's  predictions  is  only  as  good  as  the  completeness  and 
validity  of  the  set  of  individual  performance  models  underlying 
them.  Where  in  a real  mission,  for  example,  stress  is  only 
partly  a matter  of  required  tempo,  SAINT  prediction's  concerning 
task  branching  and  performance  accuracy  may  be  misleading.  The 
point  here  is  an  obvious  one  and  has  been  made  by  us  and  by  others 
in  connection  with  other  bottom-up  approaches.  The  matter  is, 
nonetheless,  of  continuing  concern. 
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2.2.4  Lockheed -Georg in 

Because  it  is  oriented  toward  establish:’’.  ; the  safety  and 
reliability  of  systems,  the  Lockhced-Georgi a technique  (L-G)  is 
concerned  explicitly  and  almost  entirely  with  the  identification  and 
assessment  of  failure  modes  in  complex  man-machine  systems. 

It  represents  an  analytic  approach  to  performance  prediction 
as  opposed  to  the  synthetic  approach  employed  by  the  three 
techniques  reviewed  above.  In  the  study  of  alternative  configura- 
tions of  men  and  equipment  in  All-Weather  Landing  Systems  (AWLS), 
from  which  the  material  for  the  current  discussion  was  drawn,  the 
technique  employs  point  estimates  of  reliability  selected  from 
data  banks,  the  human  factors  literature  and,  in  one  instance,  a 
model  of  human  scanning  and  detection  performance  (Adkins  et.al., 

1967).  The  technique  employs  the  concept  of  fault-trees  and  utilizes 
the  standard  conventions  of  reliability  theory  for  aggregation  of 
reliability  data  contained  in  those  analytic  structures. 

The  Lockheed-Georgia  AWLS  technique  is  intended  to  answer 
the  following  questions: 

"1.  Do  the  demands  that  are  placed  on  the  pilot  in  terms 
of  oral,  visual,  and  physical  requirements  at  the 
critical  phases  of  landing  and  the  multiple  task 
situations  impair  safety  or  have  disor ientative 
effects  on  the  flight  crew? 

2.  What  is  the  adequacy  of  the  visual  command  and 
position  information  as  well  as  of  malfunction  clues? 

3.  What  is  the  distribution  of  work  load  among  crew 
members? 

| I 

I 
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4.  Is  the  visual  scanning  pattern  optimum  such  that 
maximum  error-free  assimilation  of  data  can  be 
achieved?  Is  the  location  oi  instruments  and  controls 
optimum? 

5.  Are  redundancy  and/or  monitoring  provisions  presented 
in  a manner  that  accomplishes  the  desired  objective? 

Are  these  provisions  effective,  and  usable? 

As  an  example,  the  copilot  flight  direction  information 
may  serve  as  backup  for  that  of  the  pilot,  but  is  there  any 
effective  redundancy  if  the  copilot  is  looking  out  the 
window  in  an  attempt  to  establish  visual  contact 
with  the  ground  while  the  pilot  is  totally  concentrating 
on  his  own  instrument  system? 

6.  What  is  the  probability  of  inadvertent  actuation  or 
response'  ftr^a  given  task?  Also,  what  is  the  sus- 
ceptability  to  misinterpretation  or  misreading  of 
visual  displays? 

7.  What  is  the  adequacy  of  the  feedback  as  a result  of  pilot 
action  during  the  critical  phases  of  flight? 

8.  Docs  the  sequence  of  tasks  being  performed  during  the 
landing  operation  yield  maximum  efficiency  and  minimum 
error  probability  on  the  part  of  the  crew?" 
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As  in  almost  a 11  techniques  of  the  network  type,  the  first 
step  in  the  L-C.  approach  is  a comprehensive  analysis  of  the 
tasks  and  subtasks  that  must  be  accomplished,  the  time  required 
to  complete  each,  and  an  assessment  of  the  criticality  of 
successful  completion.  Since  in  the  AWLS  project,  the  displays 
and  controls  employed  in  the  tasks/subtasks  were  of  direct  concern, 
this  analysis  included  a study  of  "informational,  input"  and 
"control  output/feedback"  to  the  pilot  and  crew. 


From  this  task  analysis  and  a parallel  analysis  of  system 
hardware  and  control  functions,  fault  trees  were  constructed  that 
depicted  the  ways  in  which  the  mission  (in  this  case,  a successful 
landing  or  successful  go-around)  could  fail,  and  the  etiology, 
in  terms  of  malfunctions  of  equipment  and  incorrect  responses 
on  the  parts  of  operators,  associated  with  each  of  these  failure 
i modes.  Reliability  data  were  then  drawn  from  a variety  of  data 

banks  (e.g.,  AIR  Data  Store,  Titan  II,  Aerojet  General)  and 
posted  to  appropriate  branches  of  the  trees. 

As  noted  above,  the  degree  of  success  with  which  various 
displays  and  controls  could  be  used  by  operators  was  considered  to 
be  critical  to  mission  outcome.  Further,  it  was  cons idered  that 
degree  of  success  was  related  monofonically  to  the  location  of  the 
required  display  or  control  on  the  instrument  panel.  Given  this 
assumption  and  the  obvious  impossibility  of  locating  all 
instruments  at  the  most  desirable  location  (panel  center),  an 
empirical  model  based  on  detection  and  response  probabilities  as 
a function  of  off-axis  scanning  provided  inputs  to  branches 
associated  with  failures  in  input  ot  feedback  of  information. 


I 
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The  nominal  probability  of  occurr  nee  of  a given  failure 
mode  identified  in  the  tree  was  them  assessed  as  unity  minus  the 
product  of  success  probabilities  post  - d to  blanches  leading  to 
that  mode.  In  the  AWLS  application,  however,  it  was  important  to 
consider  that  occurrences  of  given  failures  might  only  b ••  critical 
to  success  if  they  occurred  at  certain  times  in  the  landing  sequence. 

In  such  instances,  rate  of  failure  production  (as  related  to  duty 
time,  mean-time  between  failure,  etc.)  became  important.  Where 
relevant,  then,  probability  of  success  is  defined  hy  the  standard 
reliability  equation. 

p = where  m = 1/f 

r 

and  Pr  = probability  of  success 
t = duty  time 
m = MTBF 
f = failure  rate 

There  are  many  other  facets  to  the  I,-G  approach  that  make  it 
useful  in  the  assessment  of  complex  aggregations  of  procedures,  and 
hardware.  One  is  ASSET  (Advanced  System  Safety  Engineering  Tech- 
niques) , which  contains  a large  set  of  programs  for  computing  the 
reliabilities  of  different  types  of  series,  parallel, and  redundant 
man/machine  networks  and  of  deriving  from  their  nominal  reliab.il  iti* 
an  estimate  of  the  associated  "safety"  level,  based  on  load  factors, 
approach  patterns,  ground  control  accuracy,  etc.  A second  is  t lv- 
transformation  of  historical  accident  data  into  "hazard"  in  format  ion 
that  can  bo  used  in  assessment  of  the  c-i  i t i ea  1 i t \ of  various  failure 
modes . 
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2.2.5  THERP 

Another  tree-structure  approach  to  performance  prediction 
is  the  Technique  for  Human  Error  Prediction  (THERP)  developed  by 
Swain  at  Sandia.  This  technique  has  been  discussed  in  a 
variety  of  different  contexts  (sea,  for  example,  Swain  and 
Gut Lman , 19/5;  Swain,  1964;  1973;  1974)  and  has  been  reviewed 
at  length  by  Meister  (1971).*  In  view  of  this  exposure,  we  shall 
discuss  only  what  we  consider  to  be  the  highlights  of  the  approach. 

It  is  interesting  to  note  that  the  developers  of  THERP  do  not 
consider  their  approach  to  be  a model  in  the  classic  sense: 

"The  term  'model'  has  several  meanings,  and  is  used 
loosely  in  the  behavioral  sciences.  For  our  purposes  in 
describing  THERP  (Technique  for  Human  Error  Rate  Predic- 
tion) we  will  use  the  narrow  meaning  of  'model,'  defined 
as  'a  set  of  relations  and  operating  principles. ' 

Although  this  may  slight  some  who  are  interested  in 
theory  development,  THERP  was  developed  as  an  applied 
technique,  for  use  as  a fast  and  relatively  simple  method 
of  providing  data  and  recommendations  to  systems  designers 
and  analysts  who  require  a quantitative  estimate  of  the 
effects  of  human  errors  on  system  performance. 

"THERP  is  not  a model  .in  the  usual  sense  of  a 
hypothetical  analogy,  although  it  is  regarded  as  a 
modest  form  of  mathematical  (Boolean)  modeling,  repre- 
senting behavior  variability  by  simple  equations  dealing 
with  equipment  parameters,  human  redundancy,  training, 
stress,  etc."  (Swain  & Guttman,  1975,  p.  116.) 


*An  up-t.o— date  bibliography  of  Sandia  reports  on  TIHe’P  is 
contained  in  the  Sandia  Corporation  publication,  "Description 
of  Human  Factors  by  Sandia  Laboratories , " Sand i a Corp. , 

Albuqu  !i  [ue , N w Mexico,  January,  1976 . 
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As  noted  by  Meistor  (1964),THERP  is  an  iterative  procedure 
consisting  of  the  following  five  steps  (which  may  or  may  not  be 
accomplished  and  re-accomplished  in  the  same  order,  until  system 
failures  resulting  from  human  error  are  at  an  acceptable  level) : 

"(1)  Define  the  system  or  subsystem  failure  which  is  to 
be  evaluated. 

(2)  Identify  and  list  all  the  human  operations  performed 
and  their  relationships  to  system  tasks  and  functions. 

(3)  Predict  error  rates  for  each  human  operation  or  group 
of  operations  pertinent  to  the  evaluation. 

(4)  Determine  the  effect  of  human  errors  on  the  system 
or  subsystem  failure  rate  as  a consequence  of  the 
estimated  effects  of  the  recommended  changes."  (p.630) 


As  in  all  other  network  techniques,  the  first  two  of  these 
steps  aim  to  formalize  the  tasks  under  study  and  to  yield  a 
specification  of  their  interrelationships.  The  outcome  of  these 
prior  activities  is  a diagrammatic  task  analysis  in  the  form  of 
a tree  structure  that  represents  subtask  elements 
as  nodes  and  probabilities  of  successful  and  unsuccessful 
accomplishment  as  branches.  The  data  for  these  letter  estimates 
came  originally  from  the'  ATK  Data  Store  but  more  recently  represent 
estimates  derived  from  empirical  research  conducted  at  Sand i a Carp. 
(Swain,  1974.) 
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2 . 2 . 3 . 1 Prediction  and  Aggregation  of  Error  Rates 

The  assignment  of  branch  probabilities  and  their  aggregation 
are  critical  activities  in  the  exploitation  of  THERP.  An  we  have 
said,  information  from  data  banks  represents  an  important  resource 
in  the  first  of  these  tasks,  but  the  retrieval,  selection  and 
application  of  this  information  is  indirect  and  highly  judgmental 
in  character.  Swain  (1964)  points  out  that  the  "predictions"  of 
error  rate  posted  to  branches  in  the  probability  trees  are 
ordinarily  made  by  a team  of  project  engineers,  operations 
researchers,  safety  engineers,  and  human  factors  specialists  who, 
in  the  aggregate,  are  intimately  acquainted  with  the  detailed 
functioning  of  the  system  being  modelled,  the  equipment  being 
used,  environmental  factors  obtaining,  etc., and  with  the  interac- 
tions that  may  occur  between  system  events.  Furthermore,  a 
variety  of  different  values,  from  one  or  more  point  estimates  of 
average  rate,  to  "fiducial"  points  reflecting,  for  example. 


estimates  two  or  more  standard  deviations  from  the  mean,  are 
predicted,  depending  on  the  requirements  of  the  problem. 

After  error  rates  are  determined,  estimates  are  then  made, 
again  by  experts,  that  errors  of  the  specific  type  or  class 
identified  could  result  in  system  failure.  The  probability  of  a 
failure  is  then  modelled  as  the  product  of  the  probability  ( !•'  ^ ) 
that  each  error  identified,  if  it  occurs,  will  result  in  the 
fciilure  and  tine  probability  (P.)  that  a system  operation  permitting 
the  error  in  question  to  bo  made  will  occur.* 


upon  the  application:  may  result  from  P , or  P0  wh  ;re  p and  P. 

are  human  errors.  Thus  P . - P] +P2~Pip2 • 0 

probability  of  ?i  and  P2,  where  Pj  is  an  < 

defect  or  some  other  factor  which,  when  it  occurs,  sets  u;>  a poten 
failure  condition  only  if  a human  error  (Pi)  a t 
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The  total  system  failure  rate  resulting  from  errors  in  huma 
performance  is  given  by  the  expression 

n 

Q,  = 1 - [n  (1-Q,  ) ] 
r k=l  K 

where  Q = the  probability  that  one  or  more  failure  conditions 
will  result  from  errors  in  at  least  one  of  n classes  of  errors, 
and  is  the  probability  of  one  or  more  failure  conditions  that 
exist  as  a result  of  Class  k errors  occurring  in  n operations. 


2 . 2 . 5 . 2 Assumptions  Concerning Human  Performance 

THERP  makes  a number  of  important  assumptions  concerning 
human  behavior  and  performance.  Among  them  are  the  following: 

1)  The  standard  deviations  of  the  distributions  of  error 
rates  for  two  tasks  that  are  similar  are  proportional 
to  the  means  of  the  distributions  for  those  tasks. 

2)  Where  two  operators  work  together,  the  reliability, 

R,  due  to  redundancy  is  given  as 

1 - ( 1 - R i ) (T()  i 1<  j (T2) 


where  Rj  is  the  roliabi  1 i t',  of  a single  opera ! c , t . is  t * 
percent  of  time  the  second  op  Tutor  can  observe  the  first 
and  T2  is  the  percent  of  t in:  remaining  to  oor.pl et  ••  the  t a* 

(Me is ter , 1964).  The  probabi lity  oi  one  person ' s detect in 
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3)  Denoting  the  probability  of  error  in  an  important  time  - 
critical  task  on  which  errors  occur  infrequently  as  p, 
the  probability  of  making  an  error  on  n repetitions  of 
the  task  is  equal  to  2p(n-l),  up  to  a limiting  value  of 

1.0. 

4)  For  conditions  of  little  time  stress,  the  probability  of 
error  on  the  first  repetition  can  be  approximated  as  the 
square  of  the  probability  of  error  on  the  first  trial. 
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2 . 2 . 5 . 3 Comment 

Used  by  human  factors  engineers  intimately  familiar  with  the 
design  and  operation  of  the  system  whose  performance  they  are 
attempting  to  predict,  THERE  and  the  L-G  approach  appear  to  bo 
successful  techniques.  The  concentration  of  analytic  activity 
prior  to  the  actual  construction  of  the  network  and  aggregation 
of  assigned  probabilities  serves  the  extremely  important  function 
of  aiding  the  identification  of  system  missions  that  could  easily 
be  compromised  by  unreliable  performance  of  men,  equipment,  or  thei 
interne  tion . 


It  is  important  to  note  that  there  are  a number  of  pitfalls 
associated  with  the  fault  tree  approach,  particularly  when  it  is 
employed  in  the?  context  of  an  incompletely  specified  system. 
Fussell  (1974)  has  accumulated  these  into  three  basic  categories: 

1)  oversight  and  omissionof  potential  failure  modes  and 
their  outcomes, 

2)  application  of  poor  or  inapplicable  failure  data  or  of 
incorrect  assumptions  to  highly  complex  systems,  and 

3)  failure  to  account  properly  for  mutually  exclusive  events 
that  occur  in  the  same  tree. 

To  the  extent  that  these  pitfalls  can  bo  avoided  through  the 
involvement  of  knowledgeable  persons-  1 and  with  t lie  computer  aids 
to  fault  tree  analysis  that  are  beginning  to  appear  (see,  for 
example , Vcsol y and  Narum,  1970)  , this  technique  represent  s a 
valuable  tool  for  performance  predict  ion. 
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2.2.6  Summary  of  Network  Techniques 

In  this  section  we  have  briefly  reviewed  five  performance 
prediction  techniques  that  employ  network  methods.  In  the 
aggregate,  these  techniques  illustrate  the  following: 

1)  The  use  in  analytic  and  simulation  approaches, 
respectively,  of  point  estimates  and  of  distributions 
of  task  completion  time  and  reliability. 

2)  The  use  of  data  bank  information  and/or  information 
generated  from  individual  task  performance  models. 

3)  The  use  of  three  different  concepts  of  reliability: 
one  based  on  the  ratio  of  total  required  task 
time  to  total  available  time;  a second  based  on 
empirically  determined  probabilities  of  successful 
(or  unsuccessful)  performance  with  no  explicit  time 
constraint;  and  a third  based  on  estimates  of  moan- 
time-between  failure  and  required  duty  time. 

4)  Two  essentially  different  approaches  to  the 
assessment  of  reliability.  One  of  these  is 
synthetic  in  nature,  requiring  representation  in 
network  form  of  all  tasks  and  activities  that  must 
be  accomplished  in  order  to  complete  a mission. 

The  other  is  analytic,  requiring  representation  of 
those  tasks  that,  if  accomplished  incorrectly,  will 
result  in  an  unsuccessful  mission. 

5)  The  use  of  qualitative  contextual  information  to 
sharpen  time  and  reliability  estimates  made  on  the 
basis  of  historical  flat  mm  the  oxpor  imontul 
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This  set  also  illustrates  the  difficulty  of  maintaining 
a distinction  between  "models"  of  the  type  that  form  the  major 
substance  of  this  report  and  what  we  have  chosen  to  call  "network 
techniques."  At  least  one  of  these  — SWM  — is  a legitimate 
member  of  both  categories. 

In  the  discussions  above,  we  have  made  a number  of  comments 
regarding  characteristics  of  the  network  approach.  It  is 
appropriate  here  to  summarize  these  points: 

1)  There  seems  little  doubt  that  consideration  and 
evaluation  of  a system  process  in  enough  detail  so 
that  procedure  networks  and/or  fault  trees  can  be 
constructed  is  of  value  in  highlighting  aspects  of 
performance  that  are  critical  to  mission  success. 

The  value  of  this  activity  would,  in  our  judgment, 
be  high  even  if  no  subsequent  attempt  were  m ule  to 
employ  the  networks  in  a predictive  fashion  through 
the  aggregation  of  branch  reliabilities  and/or 
completion  times.  From  this  point  of  view,  the 
network  approaches  have  the  same  general  heuristic 
value  that  hcis  been  assigned  to  the  use  of  decision 
trees  in  medical  diagnosis  (see,  for  example.  Lusted, 
1963}  and  flow  charts  in  computer  programming. 

2)  F.xcept  where  directly  relevant  experience  with 
similar  tasks  in  similar  contexts  exists,  distri- 
butions of  within  operator  and  between  operator 
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In  those  models  where  point  estimates  are 
combined  over  a set  of  subtasks,  there  are  few 
principles  to  guide  the  selection  of  a composite 
distribution,  except,  possibly,  the  Central  Limit 
Theorem.  (Even  here,  there  may  be  some  doubt  con- 
cerning the  number  of  individual  activities  required 
for  a valid  assumption  of  normality.) 

Even  where  one  has  considerable  confidence  in  either 
point  estimates  of  distributions  of  subtask  completion 
tine  and/or  probability  of  success,  the  nature  of  the 
interrelationships  among  subtasks  in  the  network  may 
preclude  accurate  prediction  of  total  task  parameters. 
The  matter  is  of  only  minor  concern  where  a single 
individual  may  perform  more  than  one  operation  at  given 
time  (e.g. , as  in  a pursuit  tracking  task)  but  is  of 
significant  concern  where  multiple  operators  perform 
in  parallel.  At  the  very  least,  one  must  be  concerned 
about  the  combinatorial  rules  used  to  aggregate  subtask 
data,  and  be  prepared  to  find  that  the  product  £*nd 
summation  rules  drawn  from  standard  reliability  theory 
do  not  afford  adequate  approximations. 

There  are  no  direct  mechanisms  within  the  models  by 
which  to  assess  such  factors  as  v.’i  thin- subtask  learning 
t ime- sharing. 
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6)  The  approaches  may  not  yield  acceptable  predictions 
when  the  observable  subtask  density  is  low  and/or  is 
composed  of  activities  which  are  largely  cognitive 
in  content. 


Two  points  must  be  made  concerning  these  alleged  difficulties. 
First,  most  can  be  addressed  experimentally  by  comparing  the  actual 
performance  of  operators  with  the  predictions  of  the  models.  An 
example  of  this  approach  is  provided  by  the  work  of  Mills  & Hatfield 
(1974).  Following  this  approach, one  could  examine  the  consequences 
for  total  task  prediction  of  network  complexity,  network  density, 
and  subtask  character  (implicit  vs.  explicit  performance)  and 
could  then  derive  appropriate  combinatorial  rules  under  different 
conditions  of  time  stress  and  operator  sophistication. 


Secondly,  one  might  find  that  certain  simplifications 

(such  as  the  assumption  of  normality  or  use  of  a product  rule)  do 

not  introduce  errors  of  sufficient  magnitude  to  invalidate  their 

use.*  As  Knowles,  et  al  (1969)  have  noted, 

"Fortunately,  the  ’if-then’  nature  of  system-level 
evaluations  based  on  analytical  models  early  in  the 
design  cycle  suggests  that  a high  degree  of  absolute 
precision  in  human  performance  data  may  not  be  a 
prime  requirement.  In  most,  cases  system  configurations 
are  compared  with  one  another  to  determine  the  host  of 
several  potentially  acceptable  designs.  In  the  rest  of 
the  cases  the  problem  is  to  arrive  at  a ballpark  estimate 
of  system  performance,  or  to  allocate  performance  r.  ■ : . » i ru- 
men ts  so  that  potential  trouble,  areas  can  bo  id  on’,  if  ml, 
or  to  set  criter  ia  for  suboptimisation  of  parts  o ' l_  1 1 
system. " (p.  581 ) . 
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2.3  Control  Thoo re ti c Mode 1 s 

By  control-theore ti c models  we  mean  those  models  that  have 
their  analytic  bases  in  control  theory  and  statistical  estimation 
and  decision  theory.  Traditionally,  these  models  have  attempted 
to  describe  human  performance  in  continuous  control  and  monitoring 
tasks.  The  emphasis  has  been  on  models  that  describe  the  human 
operator  as  a system  element  in  a feedback  control  loop.  In  this 
context,  models  exist  for  continuous  control  (manual  control), 
for  instrument  monitoring,  and  for  detection  of  and  adaptation  to 
abrupt  changes  in  a system  (such  as  would  occur  in  a failure) . In 
addition,  approaches  to  incorporating  stress  effects  in  these 
models,  especially  those  stresses  associated  with  attentional 
demand  (i.e.,  workload),  have  been  developed. 

Control-  and  decision-theoretic  models  could  be  useful  in 
the  analysis  of  various  phases  of  command  and  control  problems 
such  as  those  associated  with  RPV  control.  For  example,  the 
problems  of  monitoring  and  controlling  separation  between  RPVs, 
including  those  imposed  by  sharing  attention  in  the  control  of 
several  RPVs,  may  be  amenable  to  analysis  with  such  models. 

Of  course,  the  most  .likely  area  of  application  in  command  and 
control  would  be  in  problems  of  continuous  control,  e. g. , the 
"terminal"  phase  of  the  RPV  mission. 

Models  of  this  type  have  been  developed  largely  by  systems 
engineers  with  a prima ry  interest  in  pi ed icting  total  man-m  ichin 
system  performance.  This  history  has  had  a direct  impact  on  the 
characteristics  of  the  models  that  have  emerged.  Thus,  the  mod  'Is 
are  not  intended  to  provide  structural  analogs  of  the  human  opera: 
though  some  do  incorporate  a degree  o f structural  similarity. 

The  mode l s tend  to  describe  human  ; .•  foil  anew  at:  the  system  level 
and  In  system  ter  s . Conce  t is  re  w Lth  the  opei  ttor ' s con t '• 
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interaction  with  the  system,  as  demanded  by  closed-loop  analysis, 
than  wi  th  his  response  to  discrete  events. 

The  system  approach  used  in  manual  control  models  has  impli- 
cations for  the  level  of  detail  at  which  man-machine  interface 
problems  are  addressed.  In  the  display  area,  for  example,  the 
principle  concern  has  been  with  the  information  requirements  of 
a task  rather  than  with  the  human  engineering  aspects  of  display 
design.  Thus,  applications  of  the  models  have  generally  addressed 
the  question  of  whether  the  information  provided  to  the  controller 
is  adequate  to  achieve  requisite  performance  with  acceptable  work- 
load and,  if  not,  what  information  is  necessary.  The  effects  of 
display  layout  on  performance  and  workload  are  also  treated  (by 
the  visual  scanning  models).  However,  display  questions  related 
to  display  readability  such  as  color,  brightness/contrast , 
linear  vs.  rotary  indication,  inside-out  vs.  outside-in,  etc., 
are  largely  ignored  in  manual  control  analyses.  Though  there 
has  been  some  recent,  work  on  control  stick  design  using  manual 
control  models  (e.g.,  Levison  and  Houck,  1975),  detailed  analysis  and 
design  of  this  aspect  of  the  system  interface  lias  been  missing. 

Control-theoretic  models  generally  require  as  an 
input  a relatively  detailed,  analytic  model  of  the  "machine" 
to  be  controlled  and/or  monitored.  This  occurs  because  the 
ultimate  form  of  the  model  or  its  para • o ters  reflects  human 
adaptation  to  the  task  to  be  per formed  and,  furtheri  >re,  because 
these  models  art'  generally  appropriate  only  for  skilled  operator:; 
who  are  attempting  to  act  rationally  to  achieve  we l l -defined 
object  Lves.  Finally,  we  note  that  th<  models  have  been  applied 
principally  in  the  area  of  vehicle  control  . Only  recently  have' 
the  modi*]  structures  and  not  inns  been  • eve  lot  • ; that  su 
the  potential  tor  appl  ic.i  tion  to 


probl  i.as  in 


i i ch  human  c;  >n  t in  > I 
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actions  are  infrequent  and  in  which  raoni  tor  i nq  and  decision-mukinq 
are  the  operator's  main  activities. 

2.3.1  Models  for  the  Human  Controller 

Sophisticated  mathematical  tools  have  been  employed  in  an 
attempt  to  develop  analytic  or  computer  models  that  will  predict 
accurately  the  performance  of  human  operators  in  continuous 
control  tasks  and  a number  of  different  models  have  resulted. 

These  include,  for  example,  sampled-data  models  (e.g.,  Bekey, 

1962),  finite-state  models  (e.g.,  Fogel  and  Moore,  1968),  learning 
models  (Preyss,  1968;  see  Abstract  No.  4),  and  so  on.  Although  many 
of  these  models  have  interesting  and  useful  features,  it  is  fair 
to  say  that,  at  present,  the  manual  control  field  is  dominated 
by  models  based  on  control  theory  developments,  namely,  quasi  - 
linear  describing  function  models,  fixed-form  parameter  optimiza- 
tion models, and  state-space  optimal  control  models.  This  review 
will  focus  on  these  dominant  models.  However,  because  of  their 
potential  for  incorporation  in  total  mission  models,  simulation 
models  for  manual  control  will  also  be  discussed  (including 
simulation  versions  of  the  dominant  models) . 

2.3.1  .1  Qua q 1 1 inear  Models 

The  human  controller  is  a complex  control  and  inform  : '..ion 
processing  system.  It  is  generally  acknowledg  ’d  lh.it  ho  is 
adaptive , ti me-va rying , frequently  non-linear , and  that  his 
behavior  is  stochastic  in  nature.  Such  sysla-  . arc,  of  course, 
very  difficult  to  analyse  and  characterise.  Howev  •:  , there  ha., 
been  considerable  success  in  applying  quar.il  in  or  describing 
function  theory  to  the  problem  of  modeling  hue.  m con  ro  1 bohavioi  . 


I 
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The  development  of  quasilinear  theory  and  its  application 
to  manual  control  was  pioneered  by  Tustin  (1944),  (]94'7).  This 

work  and  that  of  Russell  (1951),  Elkind  (1956),  McRuer  and 
Krendel  (1957),  and  McRuer,  Graham,  Krendel  and  Riesner  (1965) 
resulted  in  a set  of  quasilinear  models  that  are  quite  adept  at 
predicting  human  control  behavior  in  the  simple,  but  important 
class  of  problems  involving  compensatory  tracking  with  a single 
display  and  a single  manipulator.  The  work  is  well  summarized 
in  the  excellent  report  of  McRuer,  et  al . (1965)  and  in  several 

surveys  (e.g.,  Elkind  1964;  Young,  1973).  A very  useful , 
up-to-date  discussion  (which  includes  more  advanced  aspects  of 
the  model,  including  mult  i loop  analysis)  is  the  recent  monograph 
of  McRuer  and  Krendel  (1974).  This  monograph  also  contains  a 
bibliography  of  applications  that  is  quite  extensive. 

Quasilinear  models  concentrate,  on  human  control  strategies 
and  tend  to  deal  with  in'ormation  process i rig  behavior  in  an 
explicit  fashion  only  when  overt  visual  scanning  is  an  integral 
part  of  the  task.  The  quasilinear  model  for  human  control  per- 
formance consists  of  a describing  function  that  accounts  for  the 
portion  of  the  human  controller's  output  that  is  linearly 
related  to  his  input  and  a "remnant"  term  that  represents  the 
difference  between  the  output  of  the  describing  function  and  that 
of  the  human  controller. 
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This  describing  function  is  composed  of  factors  related  to  some 
human  limitations,  namely,  reaction  delays  (t)  and  lags  attrib- 
uted to  the  neuromuscular  system  (T  ) and  of  factors  used  to 
model  the  human's  adaptive  equalization  characteristics. 


The  most  important,  and  perhaps  most  elegant,  result  of 
quasilinear  manual  control  theory  is  embodied  in  the  "crossover 
model"  (See  Abstract  No. 5)  which  relates  Y (jw)  , to  the  transfer 
function  of  the  controlled  element,  Y (jw),  by  the  equation 


Yp(jo)  Yc(ju) 


jM 


(2) 


where  m is  the  crossover  frequency  and  r an  effective  time 

delay.  The  crossover  model  is  a mathematical  statement  of  the 

observation  that  human  controllers  choose  their  equalization 

characteristics  so  that  the  closed-loop  system  dynamics 

approximate  those  of  a "good"  feedback  control  system.  The 

parameters  of  the  crossover  model , and  , or  those  of 

c e 

equation  (1),  are  generally  task  dependent;  values  for  them 
have  been  traditionally  selected  on  the  basis  of  verbal  adjust- 
ment: rules  that  are,  in  turn,  based  on  considerable  experimental 
data.  The  crossover  model  is  .intended  to  be  accurate  only  at 

frequencies  in  the  vicinity  of  m . However,  this  is  often  all 

e 

that  is  necessary  to  adequately  specify  the  closed-loop  dynamic 
chai  ictei  in;  i ;s  of  a single-loop  syst  *m.  >n  the  othei  hand,  it 
may  not  bo  sufficient  to  predict  detailed  system  performance, 
particularly  if  there  is  a substantial  remnant  component  in  the 
con t rol tor's  respons . . 
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Thus , the  crossover  model  eodifi  ;s  a large  body  of  dal  i, 
is  easily  understood,  and  permits  reasonably  com;)]  ex  single- loop 
systems  to  be  analyzed  simply  (sometimes  with  just  paper  and 
pencil).  A limitation  of  the  model  is  the  task  dependence  of 
its  basic  parameters  and  the  requirement  for  verbal  adjustment 
rules  for  their  selection.  These  adjustment  rules  work  well 
enough  within  the  domain  in  which  they  were  developed  and 
empirically  valida ted,  but  difficulties  arise  when  one  attempts  to 
predict  results  in  new  situations.  The  problem  would  be  more 
manageable  if  each  new  variable  considered  had  an  effect  on  the 
model  parameters  that  could  be  predicted  independently  of  other 
variable  changes,  but  this  is  not  usually  the  case.  Thus, 
unless  a particular  combination  of  conditions  has  been  studied 
empirically,  the  basis  for  pvc  di c ti  >:  j results  for  these  conditions 
is  tenuous;  one  is  forced  either  to  develop  a catalog  of  all 
possible  combinations  of  conditions  or  to  invent  a model,  one 
level  removed,  that  provides  formal  rules  for  setting  parameters. 

The  describing  function  approach  to  multi-input,  multi- 
output  manual  control  systems  is  based  on  the  multi-loop  analysis 
techniques  of  classical  control  theory  and  on  the  methods  and 
insights  developed  for  the  single-loop  systems.  Unfortunately, 
the  attempt  to  extend  single-loop  results  to  multi-loop  manual 
control  problems  turns  out  Lo  be  an  enormous  step  for  a number 
of  reasons . First,  there  is  the  problem  of  predict iag  what 
! - >ps  the  hui  1 1 1 w i ! ! clos<  . The)  i • a]  v.  iys  alte  tives  and , 
generally,  the  structure  cannot  be  "identified"  uniquely  f ••on; 
data  in  mu 1 1 i - 1 ( >p  situations  ( s < , e . g . , S t a p ] Cot  t c ’ t i : • and 
Tenant/  1 9 ))  . * T1  lere  fore , it  is  necess  1 < > pos  ula  < | iop 

*Whn  t dati  there  are  foi  mul  t.  i - Jo«  describing  funo*  ions  (St  npl  <»foi « 
McKuer  a r.  i M ■ • ■ ! : 1 e ; o , I lK*7  ; S t . r > 1 • • to  i , 1 , c’  v a i g md  Ton.'  ill , 1 9 19; 

Wi 1 1 ■ u ! 1 :Hu  r , 19  ! ) i i l : ;at<  ' , i r genei  ' , s i i j le-l< 

rn  I cs  si  to  app.l]  foi  "outei  1 " b >1  r 1 ■■  ;ar  i ly  foi 
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structures  and,  often,  to  employ  fixed-form  models  in  each  loop. 

Of  course,  each  describing  function  in  each  loop  will  have 
adjustable  parameters  that  must  be  specified  — and,  as  in  the 
single-loop  case,  they  are  task— dependent . Besides  being  non- 
unique for  fixed  conditions,  loop  closures  will  also  depend  on 
display  and  control  system  design  in  a complicated  fashion. 

The  determination  of  the  multiplicity  of  pi  lot  parameters  must 
be  done  iteratively.  Thus,  the  simplicity  and  ease  of  calculation 
associated  with  the  single-loop  crossover  model  evaporates,  and 
one  is  confronted  with  substantial  problems  with  respect  to  both 
prediction  and  computation. 

The  situation  with  respect  to  the  remnant  portion  of  the 
quasilinear  models  is  less  well  developed.  The  current  view  of 
remnant  in  quasilinear  manual  control  theory  is  that,  in  the 
absence  of  display  scanning,  remnant  is  due  largely  to  irreducible 
stochastic  variation  in  the  human  operator*  (McRuer  and  Krendel., 
1974,  p.  G5).  Remnant  is  not  error  in  modeling  the  deterministic 
portion  of  the  controller's  response,  although  such  errors  could 
contribute  to  remnant.  Models  for  single-loop  remnant  consist,  of 
empirically  obtained  first-order  noise  spectra  injected  at  the 
operator's  input  (McRuer  and  Krendel,  1974,  p.  34).  Fairly 
elaborate  models  for  multi-display  scanning  have  been  devoir-,  -d 
and  have  been  used  to  predict  remnant  in  multi-loop  situations 
( see  , e . g . , Allen,  Clen  *n1  and  Jex,  1970).  if  >w  tver , the  scanning 


model  does 

not  appear 

to  yield  entire 

• 1 y udoqua ' e rt 

■suits  in 

predi cling 

remnant  (M 

tohiuvi  and  hind. 

1971)  and,  1 

* 

icauso  oJ 

t he 

into  rue  t i or 

is  between 

L o : 3 sures,  . 

;<•  muling  strati 

sgies,  et  : . 

r 

appears  to 

be  d i f f Leu 

I t -to  urn-. 

*This  inter 
model  and 

pro  ta Lion 
loach;  to  a 

is  made  explicit 
i ; t hod  foi  i ro< 

i n t he  opt  i m. 
1 1 ting  r- m in ; 

vl  coni  i - 

i (nee  be)  . 

w)  . 

Report  No.  3446 


Bolt  Beranek  and  Newman  Inc:. 


Finally,  we  should  mention  the  fact  that  describin'-;  function 
models  are  frequency- domain  models  whose  data  base  is  strictly 
valid  only  for  stationary,  random  inputs.  The  describing  function 
is  not  the  pilot’s  transfer  function.  There  have  been  efforts 
to  develop  so-called  dual-channel  controllers  that  will  allow  the 
quasilinear  approach  to  be  extended  to  problems  involving  step 
inputs,  pursuit  tracking,  quasi-predictable  inputs,  etc.,  and 
these  have  had  some  limited  success  (see  McRuer  and  Krendel , 1974, 
p.  48). 

2. 3. 1.2  Fixed  Form,  Parameter  Optimization  Models 

In  an  attempt  to  overcome  some  of  the  problems  associated 
with  selection  of  parameters  of  describing  function  models 
according  to  verbal  and  empirical  adjustment  rules,  some  inves- 
tigators have  used  fixed-form  models  (FFM)  with  the  parameters 
of  the  model  selected  so  as  to  optimise  some  criterion.  Because 
the  fixed-form  models  have  the  same  or  similar  forms  as  the 
observed  describing- function  models,  they  are  generally  consid- 
ered to  be  of  the  same  class  of  models.  However,  this  view 
is  not  strictly  correct  and  is  responsible  for  some  confusion 
with  respect  to  the  interpretation  of  remnant;  in  the  FFM  case, 
the  spectrum  of  the  modeling  error  is  sometimes  considered  to 
be  the  human  operator's  remnant. 


The  most  interest  ing  of  the  FFM  approaches  war.  1 irst 
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where  workload  in  defined  in  terms  of  the  pilot  lead  time 
constants  in  the  various  loops  (an  interpretation  that  is  based 
on  describing  function  model  results).  This  expression  is  then 
identified  with  the  subjective  "rating"  of  vehicle  handling 
qualities  provided  by  the  human  pilot.  The  amount  that  each 
performance  and  workload  term  can  contribute  to  overall  rating 
is  constrained.  This  prompts  the  interpretation,  given  by 
Billow  (1969),  that  the  pilot  adapts  his  parameters  to  minimize 
a linear  combination  of  workload  and  performance,  and  "washes 
out"  exceedingly  poor  or  good  performance  in  determining  the 
rating . 

Anderson  initially  Applied  his  approach  to  data  obtained  by 

Miller  and  Vinje  (1968)  from  a simulated  VTOL  hover  task.  These 

data  include  pilot  ratings,  performance  scores, and  estimates  of 

pilot  parameters  (obtained  by  adjusting  the  para-  e ter s of  a 

fixed- form  model  to  match  the  observed  scores) . Anderson  used 

regression  analysis  to  determine  an  expression  that  related  the 

observed  pilot  ratings  to  a combination  of  measured  performance 

scores  and  derived  pilot  (lead)  parameters.  He  then  postulated 

a fixed-form  model  and  found  the  parameters  of  the  model  that 

optimized  this  expression.  The  resulting  "opti.  al"  I-TM  was  used 

to  "predict"  scores  and  pilot  ratings.  This  resulted  in  excellent 

agreement  with  respect  to  pilot  rating,*  but  th  correspondence 

between  predicted  scores  and  measured  scores  was  not  nearly  as 

good,  of  course,  the  disagreement  in  sco  "es  was  reflected  in  a 

disagreement  between  predicted  and  "r  insured"  pilot  parameter  s . 

This  inconsistency  suggests  that  matching  pile-  rating  (to  within 

i-  .1  unit.)  is  not  sufficient  to  uniquely  dotornir.  • the  pilot  model 

*Gi ven  the  variability  of  pilot  rating  data,  a " :ooci"  notch 
corresponds  to  predicting  a rating  that  is  within  on--  unit  of 
the  rat  i ng  determined  e xpe  r inton  lu  1 ly  , wh  -r<-  pivot  i ii  i ng  is 
moasured  on  a Coopoi — Harp- *r  scale  <>r  on  •- 1> > - 1 ■ ■ n (H.arpet  ami 
C'oopc*  r , 1966). 
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parameters.  Since  Anderson's  original  work , Paper  Pilot  has 
been  applied  to  a variety  of  other  tasks  to  yield  results  of  some 
promise  (see,  o.g.,  Anderson,  Connors,  Dillow,  1971;  Arnold, 
Johnson,  billow,  1973;  Stone,  1973). 

Thus,  FFi-'s  with  model  parameters  selected  to  optimize  some 
cost  function  have  been  applied  with  some  success.  This  approach 
replaces  the  verbal  adjustment  rules  of  describing  function 
theory  with  a pystoi  i tic  procedure  that  should  be  inherently 
more  "predictive."  However,  there  are  serious  problems  and 
limitations  associated  with  the  approach  that  have  yet  to  be 
resolved.  In  the  case  of  Paper  Pilot,  schemes  for  priori 
selection  of  a rating  function  are  needed  if  the  method  is  to  be 
truly  predictive.  Moreover,  the  technique  has  been  a less 
successful  predictor  of  performance  and  pilot  parameters  than  of 
ratings.  Part  of  the  difficulty  in  predicting  performance  (and, 
to  a lesser  extent,  parameters)  is  the  lack  of  a remnant  model. 

It  may  be  more  difficult  to  get  good  remnant  predictions  for  FFMs 
because  of  the  confounding  of  modeling  errors  with  remnant. 

Perhaps  the  most  serious  drawbacks  to  these  techniques 
arise  when  addressing  mult i -output , multi-axis,  multi-control 
problems.  Then,  as  in  the  describing  function  approach,  it  is 
necessary  to  postulate  possible  loop  structures  and  the  models 
for  each  loop.  This  cornpl  i en  t os  the  problem  of  choosing  a 
rating-cost  function,  increases  the  possibilities  of  modeling 
error,  an  1 increa  ;s  th>  number  of  parameters  t <>  i • opti  i red . 
These  factors  un  ' mbtedl  y jo  >pard  i/e  the  predict  i vt.  capabi.1  i ty 
of  the?  techniques  and  also  magnify  the  computet  ion.il  problems 
sign i f i cant  I y . 
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The  problems  associated  with  computing  the  optimizing 
parameters  are  far  from  trivial.  For  example,  gradient- type 
schemes,  besides  being  slow  in  convergence,  often  converge  • o 
local  minima.  This  problem  increases  in  severity  with  increases 
in  the  number  of  parameters.  In  addition,  bounds  may  be  require 
on  the  variable  parameters  so  as  not  to  violate  human  response 
capability  and  to  assure  physically  meaningful  values  (e.g., 
leads  must  be  positive) . When  constraints  are  placed  on  the 
independent  variables,  the  numerical  optimization  methods 
(gradient  projection)  become  less  efficient.  Finally,  the 
computation  time  requirements  to  attain  a maximum  (or  minimum) 
increase  approximately  as  the  number  of  parameters  squared. 

Thus,  for  multi-loop  systems  — where  each  loop  can  contain 
several  parameters  — computer  time  can  soon  become  excessive. 

2. 3.1.3  The  Optimal  Control  Model  of  the  Human  Operator 

A second  control- theoretic  approach  to  human  controller 
modeling  has  emerged.  This  approach  is  based  on  modern  control 
and  estimation  theory.  The  resulting  model  has  been  referred  to 
as  the  optimal  control  model  (OCM)  of  the  human  operator  (see* 
Abstract  No.  7),  although  many  of  the  key  features  of  the  model  deal 
with  the  human's  information  processing  capabilities  and  bohnvio 

The  human  controller  is  self-adaptive  and,  if  motivated  and 
given  information  about  his  perform  nee,  w ill  attempt  to  chai  je 
characteristics  so  as  to  perform  better.  On  the  othei  hand, 
human  performance*  is  limited  by  certain  inherent  coustrn  i nt  o- 
limitnt  ions  and  by  the  extent  t o which  the  hu  :un  un  : rst  in  is  tin- 
objectives  of  th  - task.  The.-  • obs  vat  ions  s -rve  as  tin-  b:  - s 

for  the  f unda  1 ■ 1 p t ion  uri  Ly ii  th  OCM,  namely , 

t he  we  1 I -rn  1 1 i v. 1 1 1 , \ II  t re  : 1 h.  . op-*:  1 1 ■ • will  act  i i 


a n*. 
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optiml  manner  subject  to  the  operator’s  internal  limitations  and 
understanding  of  the  task.  This  assumption  is  not  new  in  manual 
control  (c.g.,  Roig,  1962;  Leonard,  i960;  Obermayer  and  Muckier; 
1965)  or  in  traditional  human  engineering  (e.g.,  Simon,  1957, 
calls  it  the  Principle  of  Bounded  Rationality)  . The  novel  aspect  s 
are  the  methods  used  to  represent  human  limitations,  the 
inclusion  in  the  model  of  elements  that  compensate  optimally 
for  these  limitations  and  the  extensive  use  of  state-space 
concepts  and  the  techniques  of  modern  control  theory. 

Clearly,  if  the  basic  optimality  assumption  is  to  yield 
good  results,  it  is  necessary  to  have  reliable,  accurate, and 
meaningful  models  for  human  limitations.  Insofar  as  possible, 
these  models  (or  their  parameters)  should  reflect  intrinsic 
human  limitations  or  should  depend  primarily  on  the  interaction 
of  the  operator  with  the  environment  and  not  on  the  specifics  of 
the  control  task.  It  is  also  desirable  that  the  description  of 
human  limitations  be  parsimonious  and  that  it  be  commensurate 
with  the  modern  control  system  framework  that  is  being  employed. 
These  principles  have  guided  the  development  of  the  models  for 
human  limitations  that  will  be  described  below. 

There  are  several  reasons  for  choosing  a modern  control 
appr<  t to  it  k3  ling  the  hui  m controller , even  though  methods 
based  on  cl  • 1 control  t henry  haw  been  fairly  success  fu  1 . A 

principal  : at  i \ • * i ->:i  • s l he  basic  logic  of  the;  opti  i il  i ! y 
assumpt  ion . 'll.  • Lari  • appv-  inch  to  human  1 imitations  and  t h<* 
optimal  ity  as.nu.  gt  ion  suggest  a m xld  that  might  adapt  In  tr.sk 
specifications  >.nd  r<  jui  roi  r.  I s "auton.  1 1 i cal  1 y " an  1 tv*;  i h< . >a  ,U 
a subs  i diary  set  of  a Ijustis  :i  I rul  . Further,  slab-  - sp  •(•<• 
techniques  promi  se  t system.)  t Lc  a:  crouch  to  mult  i -input  , 
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multi-output;  systems  that  avoids  some  of  the  difficulties  associated 
with  the  application  of  multi-loo;  analysis  to  man- in- the- loop 
problems.  The  powerful  computable  ns  1 schemes  associated  with  these 
techniques  are  also  attractive  in  light  of  complex  monitoring  and 
control  problems  that  are  of  interest.  Finally,  the  use  of  a 
normative  model*  and  time-domain  analysis  should  facilitate 
"modular"  and  "graceful"  development  of  the  model  as  now  facets 
of  human  behavior  are  considered  and  understood. 

The  resulting  model  is  ca  stochastic,  time-domain  model  for 
the  human.  It  includes  a model  for  predicting  the  random  component 
of  human  response  and  is  not  limited  to  stationary  control  situa- 
tions. The  basic  model  is  composed  of  the  following:  (1)  an 

"equivalent"  perceptual  model  that  translates  displayed  variables 
into  noisy,  delayed,  perceived  variables;  (2)  an  information  pro- 
cessing model  consisting  of  an  optimal  estimator  and  a predictor 
that  generate  minimum  variance  estimates  of  the  system  sta to  from 
the  data  concerning  perceived  variables;  (3)  a set  of  "optimal  ga  ins" 
chosen  to  minimize  a quadratic  cost  functional  (a  generalization  of  t he 
mean-squared  error  criterion  that  expresses  task  requirements);  and  (4)  an 
equivalent  "motor"  or  output  model  that  accounts  for  "bandwidth" 
limitations  (frequently  associated  with  neuromotor  dynamics)  of 
the  human  and  his  inability  to  generate  noise-free  control  input:.. 

The  time-delay,  observation-  an  1 rotor-noises, and  the 
neuromotor- lag  matrix  account  for  inherent  l ini  tat  ions  on  hu.  urn 
processing  and  perceptual -motor  ac ‘ ivlty . Methods  for  choos i nq 
values  for  these  quantities  have  kr-'T.  determined  by  matching 
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experimen tal  data  and  these  values  have  been  found  to  be  generally 
independent  o f task  parameters , (Kie  Inman,  Baron  and  Levi  son,  (19/0). 
The  observation  noise  is  a key  feature  of  the  model-  rt  is,  essen- 
tially, a lumped  representation  of  human  randomness.  From  the 
standpoint  of  classical  quasilinear  describing  function  theory , 
the  observation  noise  may  be  thought  of  as  a model  for  controller 
remnant.  On  the  basis  of  considerable  experimentation,  a 
relatively  simple  set  of  rules  for  predicting  remnant  has  been 
found  (Lcvison,  Baron  and  Kleinman,  1969)  . 

The  optimal  estimator,  predictor,  and  gain  matrix  represent 
the  set  of  "adjustments"  or  "adaptations"  by  which  the  human 
attempts  to  optimize  his  behavior.  The  general  expressions  for 
these  model  elements  depend  on  the  system  and  task  and  are 
determined  according  to  well-defined  rules  by  solving  an 
appropriate  optimization  problem.  The  solution  to  the 
optimization  problem  yields’  predictions  of  the  complete  closed- 
loop  performance  statistics  of  the  system.  Probability  densities 
of  all  system  variables  (states,  outputs, and  controls)  are  generated 
as  functions  of  time,  along  with  mean  and  rms  error  deviations 
from  a nominal  path.  Moreover,  the  densities  of  the  operator’s 
estimates  and  ostim.li'-.a  errors  are  also  predicted  as  functions 
of  time.  All  computations  are  performed  using  covariance  pro- 
pagat  ion  ii.etlv  -ds,  thus  avoiding  costly  Monte  Carlo  simulations. 
However,  i ' desire  I,  a "s  imple"  or  simulation  version  of  the 
model  is  p is;  ibl  (so.  be  .low)  . 


Yh  opt  i i.1  control  model  has  been  sub  jc  'tea  t<  extensive 
valid  t i i wi  ex 1 1 . ■ 1 •,  • ■ i ' > • . > v ' • ig i ng  results.  It  h b ei 

i 1 . 1 *d  In  rclati'  ;J  y simple , st  it  ionary  nt.r<  I isks  (Kle.ii  n , 

Bai  >n  and  Levison , 1970)  and  in  m e co  pi  ox  1 isks , 1 > h t i Loi 
( ir ; i m d . i ! . , 1970)  a id  ion  s tat  : • 1 . i i • . (Kl<  nt  i i it  1 Baron , 197  i; 
Klelnr  m and  Perkins , 1974;  Klein:  in  and  Kill  • -.v.-arth , 1971; 

Raiwi  and  bevison,  1971). 
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The  optimal  control  model  of  the  human  operator  appears  to 
have  several  advantages.  First  of  all,  th<  re  has  been  considerable 
success  in  isolating  human  limitations  so  that  they  are  essen- 
tially task  independent.  That  is,  the  parameters  of  this  model 
that  describe  human  limitations  do  not  vary  significantly  with 
vehicle  dynamics  and  forcing  function  bandwidths  so  long  as  the 
underlying  linearizing  assumptions  remain  valid  and  so  long  as 
the  operator  is  motivated  to  perform  at  his  limits.  In  those 
cases  where  performance  does  not  degrade  substantially  when  the 
operator  does  not  devote  his  full  energies  to  the  task,  the 
parameter  values  adopted  by  an  operator  will  in  all  likelihood 
depend  on  subjective  factors  (such  as  psychological  set) ; 
however,  in  these  cases  performance  predictions  using  the  model 
will  remain  reasonable  and  the  model  itself  can  be  used  to 
determine  an  appropriate  range  of  operator  parameters  (Levisjrg 
Elkind  and  Ward,  1971;  Levison,  Baron  and  Junker,  1976). 

When  the  characteristics  of  the  display  or  control  mani- 
pulator are  changed,  the  approach  in  the  OEM  is  to  model  these 
changes  directly,  where  possible,  rather  than  to  adjust  para- 
meters of  the  model  according  to  some  other  rules.  This  has 
been  accomplished  in  the  case  of  visual  thresholds  (Kleinman  and 
Baron,  1973)  and,  more  recently,  in  the  case  of  control-manipulator 
dynamics  (Levison  and  Houck,  1975).  The  resulting  changes  in 
the  pilot's  describing  function  and  remnant  arc  then  predicted 
as  an  output  of  the  OCM.  This  constitutes  a fundamental  difference 
from  the  describing  function  approach  wherein  the  result  in  a chan.j'-:; 
in  describing  function  and  remnant  are  generally  catalogued  (,;e  •, 
e.g.,  Jo:,  Allen  and  Magdalono,  1971). 
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In  the  OCM,  there  in  no  need  to  postulate  loop  closures  or 
model  forms  for  the  various  loops.  Instead,  one  must  specify 
what  is  displayed  to  the  pilot  (this  is  generally  given)  and 
the  cost  function  to  be  optimized  (see  below) . The  OCM  adapts 
to  changes  in  system  dynamics  and  forcing  function  changes  via 
the  solution  to  a well-defined  optimization  problem  that  admits 
a closed- form  solution. 

' 

Another  feature  of  the  OCM  is  that  a simple  and  effective 
model  for  remnant  is  incorporated.  This  has  permitted  accurate 
predictions  of  remnant  and,  therefore,  performance.  Moreover, 
the  interference  model  (Levison  et  al . , 1971)  allows  for  accurate 
remnant  prediction  in  multi-axis  tasks  that  do  not  involve  overt- 
scanning;  because  of  the  close  connection  between  attention- 
sharing and  scannmy,  it  is  possible  that  scanning  remnant  can 
also  be  predicted  accurately. 

The  problem  mentioned  most  often  in  connection  with  the  OCM 
is  the  selection  of  a cost  functional.  In  applying  the  model  to 
predict  performance,  it  is  assumed  that  the  human  operator  will 
perform  the  control  task  so  ns  to  optimize  a quadratic  functional 
of  the  system  variables.  This  functional  is  defined  objectively 
by  specifying  relative  penalties  for  various  system  errors;  this 
i specification  involves  a careful  analysis  of  the  problem.  However, 

one  cannot  be  certain  that  human  controllers  will  optimize  the 
objective  functional  rather  than  some  subjective  criterion. 
Initially,  it  was  believed  that  this-  was  an  area  where  artist  t y 
in  applying  the  model  was  needed  (Young,  1973)  but  experience 
with  the  model  Iris  rev-  rled  few,  if  any,  instance.;  where  the 
choice  of  performance  criteria  has  actually  postal  serious 
d i f f i.cul  l ies.  Nov  • r the  1 ’ s s , t h • ' sp  a;  i { i cation  o ■ the  ms  l 
functional  remains  the  primary  area  involving  th  judgement  of 
the  .system  indy.;!  in  ,r  • lyhia  Hr  a . 
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The  information  process  In  ••  portion  of  the  OCM  include:!  an 
estimator  and  a predict© ; . so  :•!  o.  .ants  contain  perfect 

"internal " models  oi  the  s . . While  I - noti  n that  the 

trained  human  oj  ratoi  h in  Lnt  :rnal  i ie]  Ls  juite  appealing, 
the  assumption  that  the  model  is  perfect  appears  to  be  less 
wel  L justified.  To  some  extent,  model  i.  ; erf  ictions  are  account  • i 
for  by  the  observation  and  (especially)  motor  noises.  Moreover, 
results  to  date  have  indicated  that  so  lone:  as  there  are  random 
disturbances  of  any  significance,  errors  introduced  by  the 
assumption  of  a perfect  model  tend  to  be  negligible.  Tn  other 
circumstances,  however , the  perfect  model  assumption  may  lead  to 
inaccuracies  in  model  predictions. 

A related  problem  concerns  the  increased  order  of  the  pilot 
: >del  with  increase:;  in  probl  i coi  plexity.  Modern  control  theory 
(and  the  OCM  approach)  treat  input  shaping-,  sensor-,  actuator-  01 
display-dynamics , by  s tat  ntation . The  OCM  will  havi 

perfect  representation  for  th-  so  dynamics  ar.d  will  increase  in 
order  accordingly.  This  incj  ses  the  c itationa]  xpense 
obi  lining  a solution  and  sugges  s h . 1 >ii  bility 

some  reduced-order  modeling. 

Ano the r probl  coi  sms  the  i tor  j i the  model . 

This  model  may  need  furt.ivr  ref ineren t, es:  c.i  illy  with  respect  to 

ti-axis  conti  tasks . * Presi  1 y , t ; . iccountincj  fo 

Inadvertent  control  ci  >ss i s . In  iddi  n , the  i 1 

signal  r fio  has  been  used  la-  ely  is  a sur  r>  • m*  • • f o ; Lnt  i rnal 
i. to  :• ; i irnpar  feet  ions  (Bn  : >n  , t a 1 . , 1 9 7 0)  . ' re  ; i dual  irotoi 

s , , tha t ' I seal  w :on 1 1 • , ! II 

rpo rated  *"0  account  ; mot  t!  Lds  : 

e::P' > r imen  mil  evi  done’  show  ins  'hat  os  it  rol  K w r 1 1 i nt  reduce* 

in'w  ,,n  un  sy. 
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Finally,  1 L go ro us  schemes  to  es  t ■ ■ , 1 1 ■ ■ . . I stablish  confidence 
limits  f 03  th< i underlying  mode]  parai  ■ ‘ ■ .■  a of  th  ( M do  not 
presently  exist.  Attempts  to  identify  the  parameters  h.ive  run 
into  soma  difficulty  and  it  has  been  claimed  that  the  parameters 
of  the  OCK  are  not  uniquely  identi  fiable  (Phatak , V7e inert  and 
Segall,  1974).  It  is  clear  that  some  problems  can  arise  i n formal 
procedures  because  what  is  optimal  with  respect  to  a particular 
cost  functional  and  set  of  parameters  is  apt  also  to  be  optim.  il 
with  respect  to  a slightly  different  cost  functional  and  para- 
meter set.  On  the  other  hand,  the  heuristic  methods  that  have 
been  used  have  yielded  parameter  estimates  that  predict  now 
resul  ts  success  ul  ly . 

2.3.1  .4  Siaviul a t ion  Mod'.  ■ 1 s 

As  our  definition,  we  take  a manual  control  simulation  model 
to  be  a model  which  produces  a specific:  time-history  for  tin-  human 
controller's  output  given  a specific  time  history  foi  the  input;.* 
All  manual  control  models  describe  the  input /output  charac t or i s f j.cs 
of  the  human  operator  in  a manner  that  is  consistent  with  an 
Lnstant-by-instant  : :s  •••  i Lon . !!■  >w< ;ver , t h >d  ds  that  have 

ited  1 1.  ■ field  (see  previous  sect  ions)  have  g<  u 
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analysis;  that  inter-subject  difference:;,  «is  well  as  the  stochastic 
behavior  within  individuals,  makes  prediction  of  a particular 
response  essentially  impossible;  the  computational  efficiency  of 
linear  analysis  techniques;  and,  finally,  that  simulation  models 
frequently  degenerate  into  a "fiddler's  paradise." 

On  the  other  hand,  there  are  some  strong  motivations  for 
simulation  modeling.  If  one  wants  to  integrate  a manual  control  model 
with  another  model  this  will  often  require  a simulation  represen- 
tation. Certain  types  of  system  or  operator  nonlinearities  may 
not  be  amenable  to  linearization  techniques  and  will  require  simula- 
tion for  analysis.  Statistical  analyses  can  sometimes  obscure 
certain  important  response  patterns.  Finally,  analyzing  indi- 
vidual responses  may  ease  the  interpretation  of  results  and  the 
validation  of  models. 

There  have  been  many  simulation  models  developed  and  reported 
in  the  manual  control  literature,  often  differing  from  one  another 
in  a minor  fashion.  As  might  bo  expected,  this  development 
reflects  the  developments  in  linear,  analytic  models  (which,  in 
turn,  reflect  advances  in  control  theory).  Thus,  early  simula- 
tion models  include  transfer  functions  that  frequently  derive 
from  quasilinear  models  while  later  ones  arc  reminiscent  of  Fl’M's 
or  the  OCM.  Here,  to  give  a flavor  of  the  approaches  to  simulation 
modeling,  we  shall  discuss  some  represent ati ve  models. 

One  of  the  first  attempts  at  simul  1 1 j on  rod  dling  of  human 
pilots  was  conducted  by  I)i mantides  (1938).  The  simulation 
model  was  constructed  from  analog  computer  element,  and  was  com- 
posed of  elements  thought  to  be  representative  of  hum  in  line.ai 
and  nonlir  >r  behavior.  Th  , thee  nt  i ; .on  ■ r del  ineorpor  it  ••  • 
neuroma  soul  dyri  tinier;,  ic  ct  ion  <!•  lay.  aid  equ  1 1 i z.it  ion 
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transfer  functions  similar  to  those  of  quasilinear  models.  It 
also  included  direct  perception  of  derivatives  of:  displayed 
signals  and  a number  of  nonlinearities  (perceptual  and 
indifference  thresholds,  anticipation  and  dither  circuits, and 
saturation  elements).  The  parameters  of  the  "analog  pilot" 
were  varied  until  the  simulation  model  exhibited  responses  that 
were  quite  similar  to  those  of  actual  pilots.  The  model  proved 
to  be  interesting  and  instructive.  It  reflected  what  was  known 
from  quasilinear  theory  and  the  prevailing  technology  of  using 
analog  computers  to  analyze  complex,  nonlinear  control  systems . 

Costello  (1968)  developed  the  so-called  "surge  model" to 
predict  operator  response  to  high  bandwidth  inputs.  The  surge 
model  has  two  modes:  for  "sufficiently  small"  error  and  error- 

rate,  a linearized,  constant  coefficient  model  is  used;  when  the 
error  and  error-rate  exceed  pre-dete rrnined  bounds,  and  whenever 
the  phase  plane  trajectory  crosses  a switching  line,  a pre- 
programmed double  step  or  "surge"  is  introduced.  The  response  of 
the  "surge"  model  to  step  .inputs,  square  wave  inputs  and  chop,  i 
sine  wave  inputs  compares  favorably  with  responses  to  these 
inputs  observed  experimentally,  more  so  than  the  response  of  t he 
simple  linear,  constant  coefficient  portion  alone.  Costello's 
model  is  one  of  many  simulation  model:;  (c.g.,  Johansson,  1 9 7 P , 
see  Abstract  No.  8;  Plialak  and  Bokry,  1969,  s<  ■ Aha!  tact  No.  9; 
Veldhuvsen,  1976 , ; e Abstract  No . 10 : th  t employ  "d  a i : • : < > ■ i 
surfaces"  in  the  ph  . pi  n<  to  change  i 1 o operation. 

A ic-e-nt,  amb.i  ! ions  dovJepmeni  in  :*,  < : iu  I Lien  n ide  I i n j w 
undert  then  by  Onr;t«  ft  (1974,  see  Abatra-M  II).  II  i ■ del 

i nco  rporates  fixed  form  pi  lot  1 (dels.  . " ui  icy  f unc  " is 

defined  to  determine'  when  t ho  mode  I nh  nil  d "swi  tch  at  t < -nt  ion" 

■ < ■ ■ . ixi  s to  her . 1 1 i Li  1 Ln  aci  nf  I 

re:  l i:  , i t » 1 - • • . i • , i ' • > . • > 
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cross feeds.  Onstott  claims  close  agreement  between  model  and 

experimental  data, though  the  results  given  in  the  above  cited 
reference  appear  somewhat  mixed.  Again,  this  model  reflects 
the  trends  in  analytic  modeling,  particularly  the  use  of  fixed- 
form  pilot  models  with  parameters  selected  to  optimize  some 
performance  criterion. 

We  close  this  discussion  by  reemphasizing  the  point  that  the 
analytic  models,  though  not  developed  for  simulation  purposes, 
can  serve  as  the  basis  for  simulation  models.  We  have  seen  that 
many  simulation  models  employ  transfer  functions  that  are 
similar  to  the  describing  functions  developed  in  quasilinear 
theory.  (This  is  done  because  the  form  of  the  describing  function 
is  reasonable  cind  despite  the  fact  that,  from  a theoretical  stand- 
point, the  describing  functions  are  not  strictly  valid  for  this 
purpose) . It  should  also  be  mentioned  that  the  Optimal  Control 
Model,  described  earlier,  can  be  used  as  a simulation  model 
(Kleinman  and  Perkins,  1974).  The  simulation  version  of  thn 
OCM  requires  selection  of  a particular  realization  of  the  various 
sources  of  problem  randomness  (both  system  and  human)  to  generate 
the  simulation  "histories."  Since  the  model  is  a time-domain 
model  and  admits  time-varying  parameters , it  can  be  quite  general 
in  its  simulation  extension  while  retaining  features  verified 
earlier.  On  the  other  hand,  a generalized  simulation  versi  n of 
the  OCM  has  not  been  validated  experimentally. 

2.3.2  Mode  1 s for  Visual  Scanning 

In  the  manual  control  field  much  of  the  i.  . d . foi 
analyzing  and  modeling  visual  scnni  | b havior  ha:  1 en  pi 
by  the  a i re:  raft  control  problem.  A 1 iije,  but  lieu;  t ! y i »•  i.»n' 
da  fa  base  exists  with  res]  >•  • 1 to  m >. ! t < »i  I : mat*  > : . t r u mi  I 
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panels  (see  Barnes,  1972,  for-  an  analysis  o£  a significant  port  ion 
of  aircraft  eye-movement  work).  Several  models  for  predicting 
and/or  describing  scanning  behavior  have  been  suggested.  These 
models  have  their  roots  in  information  theory  and  in  statistical 
estimation  and  decision  theory.  All  the  models  we  discuss  here 
arc  relevant  to  pure  monitoring  problems.  They  differ  in  their 
theoretical  bases  and  in  the  degree  to  which  they  consider  the 
effects  of  closed-loop  control  on  scanning. 

The  first  quantitative  model  for  describing  pilot  sampling 
behavior  was  developed  by  Senders,  (1964,  Abstract  No.  .1  2 ) . 

This  model  v/as  based  on  information- theoroti  c ideas,  particularly 
Shannon's  sampling  theorem.  A basic  assu:  gtion  of  the  model  was 
that  the  human  observer  samples  the  various  signals  periodically 
and  attempts  to  reconstruct  the  time  functions  presented  on  each 
instrument.  Moreover,  it  was  assumed  that  the  operator  was 
effectively  a single  channel  device  capable  of  attending  to  only 
one  signal  at  a time.  With  these  concepts  as  a starting  point, 
Senders  was  able  to  derive  expressions  for  the  frequency  and 
duration  of  samples  of  an  instrument  given  its  input  signal 
characteristics  and  the  required  precision  of  readout.  This  model 
predicted  quite  well  the  average  of  subjects  in  an  exp  rimontil 
situation,  but  the  agreement  obtained  was  somewhat  fortuitous  in 
that  it  depended  on  a unique  experimental  condition  (Senders , 
et  al . , 1 969) . 

However , it  scored  re  is  -nobly  cion  that  I he  simple  pet  iodic 
sampling  model  would  not  adequately  prod  i cl  behavi  n in  com  pi  • 

situations , espec  ial  1 y si  nee  observed  data  ja  vi  • ■ ■ : < le  :o  ol  a peril  ; • 
r.ampl  inq  behavior  . Tak  i ng  a cue  f rim:  the  j act  t bat  piin'-,  o { t - mi  .in- 

i of  ext  rem  readings  i a * lv  i than  with 


I 
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si  jnal  reconstruction,  Senders,  ot  al.  (1966)  proposed  a condi- 
tional sampling  scheme  that  v;ould  result  in  aperiodic  behavior. 

In  thin  approach,  the  human  monitor  is  considered  as  a channel 
lor  the  transmission  of  discrete  messages  and  not  as  a channel 
for  the  transmission  of  a complete  time  function.  Then,  it  is 
possible  to  postulate  several,  not  necessarily  mutually  exclusive, 
sampling  strategies  (e.g.,  sample  when  the  probability  that  the 
signal  exceeds  a prescribed  limit  is  greater  than  some  subjective 
probability  threshold  or  when  the  probability  of  exceeding  the 
limit  is  a maximum)  (Senders,  ct  al.,  1966). 

Smallwood  (1966)  (Abstract  No.  13  ) developed  a model  which, 
although  similar  in  some  of  the  details,  v.Tas  conceptually  quite 
different  from  those  previously  developed.  His  model  was  based 
ox  two  underlying  assumptions:  (1)  the  human  operator  bases  his 

state  of  information  about  his  environment  upon  an  internal  model 
of  this  environment,  the  model  being  formed  as  a result  of  past 
perceptions  of  his  environment; ( 2)  the  human  operator  behaves 
optimally  with  respect  to  his  task  and  his  current  state  of 
information  within  his  physical  limitations.  To  apply  the  model, 
Smallwood  makes  further  assumptions.  First,  he  postulates  a fori, 
for  the  internal  model  that  describes  the  monitor's  conception 
of  the  environment  he  is  monitoring.  Smallwood's  approach  to  this 
problem  is  to  assume  that  the  monitor's  model  of  each  instrument 
is  a good  approximation  to  the  true  situation.  (Note  the 
rese  blance  1 < - the  Kalman  f i 1 ter ing  approach. ) Small w< >od ' s i ntei  ■ 
pret  at  ion  o : optimality  was  that  t he  human  noni  t.or,  interested  in 
detecting  i m-.  td  i tte  excursions  of  the  instruments  beyond  Lh  • 
threshold,  switches  his  a* tontion  to  that  instrument  for  which  t he 
probability  >f  exceed  i nq  the  threshold  is  a ir  iximu  . Thus,  he 
d ’alt  st  r i 'fly  v.-i  t h monitoring  , ess  *:»tia  I ly  i qnn  r i »v.j  t lr  ■ cunt  i o! 
ob  i cc  t i Vi 1 s . 
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The  above-merit  ionod  inode!  s have  several  limitations  that 
arise  either  from  inherent  factors  or,  in  the  case  of  Smallwood's 
model,  from  a failure  to  exploit  the  full  potential  of  the 
approach.  One  significant  limitation  was  that  none  of  the  models 
accounted  for  correlation  between  signals  on  various  indicators. 
It  is  clear  that  with  an  appropriate  choice  for  an  internal 
model,  a method  for  dealing  with  coupling  is  possible  within  the 
framework  of  Smallwood's  model,  but  he  did  not  attempt  to  include 
such  effects.  A second  significant  limitation  of  these  models  is 
that  none  of  them  take  explicitly  into  account  the  inter. action 
be  tween  sampling  and  control,  behavior.  Tn  all  cases  the  human 
operator  is  considered  as  a monitor  only;  the  control  requirement 
play  little,  if  any,  role  in  the  selection  of  a sampling  strategy. 
It  is  true  that  a po.'  ‘ '•  rio r-z  signal  analysis,  which  is  necessary 
to  obtain  the  parameters  of  the  conditional  sampling  models,  will 
include  the  effects  of  control.  However,  this  is  quite  different 
from  using  the  control  task  in  an  attempt  to  predict  sampling 
behavior . 

AL1  of  the  models  consider  the  human  operator  as  a single- 
channel  device  capable  of  processing  information  from  only  on: 
instrument  at  a time.  This  approach  is  at  variance  with  the 
experim  ntal  data  obtained  by  Lovison  and  l’lkind  (1  067), 
concerning  peripheral  tracking.  Their  experiments  showed  that 
under  c;  . tain  conditions,  pilots  could  perforin  two-axis  coi  : uisa- 
tory  {■  1 ek  i ng  '.'on  though  the  i nforr.it  ion  concerning  one  of  t h ■ 
axes  .ilv.nys  in  the  p t ipheral  visual  field.  V.'h  i 1 e the  re  mi  gh 

be  SO;  ■ argum-sit  as  t.o  whether  the  pilot  is  actually  processing 
both  s.  i <:;•  il ;;  s i • ul  l a i : ■ sis  1 y , it  seem: ; f a i rl  y ch>  :t  t h . *.  1 1 

. b > 1 i t y to  i r >rm  ; oripho  a l trucking  w i 1 ! in  f i • 1 a.  fleet  the 
pilot's  s.  , : I i . b ' i i v i o and  should  t . h < ' : • f o i * b ..aoiU'il  • i tor 
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that  no  risk  or  cost  structure  has  been  incorporated.  Equal  cost; 
are  assigned  to  all  instruments.  Moreover,  there  is  no  cost 
assigned  to  taking  a sample  or  to  switching  attention. 


Carbonell  (1966,  Abstract  No.  14)  attempted  to  overcome  sonr 
of  the  above  limitations  by  developing  a model  of  visual  sampling 
which  had  its  roots  in  queuing  theory.  If  the  human  is  assumed 
to  be  a single-channel  processor,  then  one  is  led  to  the  notion 
that  the  various  information  sources,  i.e.,  instruments,  quou-'  up 
and  wait  their  turn  to  be  processed.  The  analysis  of  scanning 
can  then  be  approached  as  a problem  in  queuing  theory  and  one 
can  arrive  at  estimates  of  the  probability  distribution  of 
simultaneous  demands,  the  probability  distribution  of  waiting 
times, and  estimates  of  the  probability  that  events  of  interest 
will  be  missed. 


Carbonell 's  model  is  clearly  more  general  and  more  flex 
than  the  models  previously  discussed.  In  addition,  it  lias 
achieved  good  accuracy  in  a validation  study  of  approach  to 
landing.  Nevertheless,  the  model  has  not  removed  all  the 
previously  cited  limitations.  Peripheral  processing  is  not 
accounted  for;  indeed,  the  concept  of  the  human  opera  to-'  be  i i 
a single-channel  processor  is  central  to  the  idea  of  the  inn! 
men l queue.  Also,  coupling  among  instruments  was  not  includ 
in  the  model,  although  Carbonell  claimed  that  such  coupling 
could  be  incori  >rated . Finally , it  should  be  pointed  out  th 
the  model  incurs  a heavy  price  i ; analytic  corn;  loxily  foi  th 
flexibility  that  is  obtained.  It  appears  that  only  through 
extensive  simulat  ion  can  one  obta  in  the  pert  i non t model  pa i . 
meters  and  predict  human  sampling  behavioi . 
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Al  Lon,  Clement,  and  Jex  (1970)  (see  Abstract  No.  L j)  and  McRuer  et  el. 
(1968)  have  attempted  to  synthesize  the  concepts  of  Senders'  scanning 
model,  multi-loop  describing  function  theory,  and  some  ideas  of 
Clement  concerning  human  signal  reconstruction  into  a theory  for 
displays  in  manual  control.  Allen,  Clement,  and  Jex  suggest  a 
model  for  scanning,  sampling, and  reconstruction  that  comprises: 

(1)  a quasilinear,  random-input  "perceptual  describing  function," 
which  multiplies  the  human  operator's  continuous  describing 
function  (i.e.,  is  inserted  serially  in  the  control-  loop);  and  (2) 
a broadband  sampling  remnant,  which  adds  to  the  basic  remnant, 
and  is  described  as  a wide-band  observation  noise  injected  at 
the  pilot's  perceptual  input.  The  "perceptual  describing  function" 
is  usually  specified  by  an  attenuation  factor  and  an  equivalent 
sampling  delay  (T  ) . The  values  of  K.  and  T depend  on  scanning 
interval,  dwell  fraction,  weighting  of  rate  information,  and  type 
of  signal  reconstruction  used. 


The  approach  of  Allen,  Clementr  and  Jex  ameliorates  the  major 
difficulty  associated  with  "open-loop"  scanning  models  in  that 
control  requirements  are  taken  into  account,  albeit  in  a quali- 
tative way.  In  addition,  the  approach  has  been  partially 
validated  in  extensive  simulation  studies.  However,  the  scanning 
model  accentuates  the  problems  already  noted  lor  the  describing 
function  approach  to  pilot  modeling.  One  must  face  the  problems 
of  selecting  multi-loop  s true  tui  s via  the  "adaptive  feedback 
selection  hypothesis"  and  also  tire  problem  of  d siding  amorr 
various  signal  reconstruction  schoj  s . Mon  o\  r , the  who] e 
process  is  iterative  in  that  scanning  pi  edict  ions  must  be  m ; i • 
after  the  control  structure  is  eh  ■ f i a.  al , but  t . 1 1 ■ * scanning  str  tteqy 
can  affect  Lire  con  trol  structur  - • . Hoi  : i un  , Cl  n t , . m.i  R 1 1 h ji  ! t 
(1971)  have  developed  a teohni-  that  yield.;  a non-  iterat  ive 
procedure  for  the  case  wh  to  li  a • is  a siusli  pi  ip  try  inst  ruiipnl  , 


. -nt  , 


i.e.,  a : i fit  dir-  rotor , need' 


R . t No . 3446  B<  > ! : Beranek  md  N< :wman  In 

the  approach  could  remain  extremely  difficult  to  use  and  computa- 
tionally inefficient . Another  important  problem  is  that  the  model 
for  scanning  remnant  does  not  appear  to  yield  entirely  adequate 
results  (Machuc-i  and  Lind,  1971)  . Accurate  remnant  representa- 
tion is,  of  course,  very  important  for  predicting  performance 
with  scanning  because  this  is  invariably  a high  remnant  situation. 

Baron  and  Kleinmari  (1968)  have  proposed  a scanning  model 
(Abstract  No.  16)  for  incorporation  in  the  GC-:  and  have  modified 
it  and  applied  it  in  a VTOL  hover  task  (Baron,  Kleinman,et  al. , 
1970) . The  basic  assumption  in  this  model  is  that  the  pilot 
chooses  his  control  input  and  his  scanning  strategy  to  minimize 
the  quadratic  cost  functional  describing  his  task  requirements. 

The  scanning  strategy  is  defined  by  a sot  of  parameters  (average 
dwell  time,  scan  frequency,  etc.)  which  in  turn  are  adjusted  to 
minimize  closed-loop  cost.  This  model  is  therefore  suitable  for 
prediction . 

There  are  several  advantages  of  this  approach  to  scanning 
prediction.  The  sampling  model  is  such  that  the  human's  moni- 
toring behavior  depends  upon  the  control  requirements  and  control 
actions  in  an  explicit  way.  Control,  and  scanning  strategics  are 
determined  at  essentially  the  same  time;  one  avoids  the  inherently 
iterative  and  judgmental  procedures  involved  in  a process  that, 
requires  the  specific  loop  topology  to  be  known  before  computing 
scar.:  i ng  para  t s . Also , one  u rids  separate  is  sumptions 
< s ri  ng  i ‘construction  < > : th  sampled  si  fnal ; the  ; ill  m estii  ito 

i > this  f . :tion,  as  In  1 1 i< 1 no  . :an  Lng  ca  . . Fii  . 1 : , the 

observed  effects  of  scanning  arc:  a natural  consequence  of  this 
approach.  Increased  average  observat  : < i nr»i. result  from  t 1 i 
impossibility  of  fix, at  i ng  fovea!  ly  ill  ; i r.p.l  ays  at  once.  This 
let’s  to  i net  vi.;  . in  i . > u 1 , ae<  r u • , r : i • I by  r , ! • t i on  . in  pi  lot 
gain  md  an  apparent  i nero  so  in  t.  i r » rb-lay. 
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The  principal  difficulty  associated  with  th  ■ OCM  approach 
would  appear  to  be  the  computational  burden  in  solving  the  option 
sampling  problem.  In  the  two-display  case,  numerical  search 
techniques  are  adequate  and  not  too  expensive.  For  more  than 
two  displays,  a suitable  optimization  algorithm  is  necessary 
and  the  analysis  is  likely  to  be  expensive,  but,  perhaps,  not 
more  so  than  other  approaches. 

A final  comment  concerning  scanning  models  seems  appropriate 
Experience  seems  to  indicate  that  choice  of  a scanning  strategy  i 
highly  idiosyncratic  and  that  there  is  great  variability  in  scan 
patterns  among  subjects.  For  example , Machuca  and  Lund  (1971), 
in  describing  the  results  of  a study  to  verify  the  model  of 
Allen,  Clement,  and  Jex, state  the  following: 


"The  pilot-subjects  adopted  different  average  scanning 
and  sampling  strategies.  Subject  preferences  in  display 
orientation,  control  sticks,  and  simulation  techniques 
were  noted  to  influence  scanning  behavior." 


and 


"A  subject  may  adjust  his  scanning  s 1 rat  igy  (i.e., 
sampling  interval  and  dwell  fraction)  to  maintain 
acceptable  control  over  the  control  tasks.  During 
this  experin  :nt,  scanning  pai  imete  - , wh  Lch  at  :mp1 
to  account  for  changes  in  scanning  • of  rmance , di 
not  yield  the  predicted  range  of  values  who  i the 
controlled  dynamics  were  changed  and  when  sirnni  it  ion 
techniques  were  altered  from  those  used  by  oilier 
exper  i mentors.  However,  in  spite  o si.  T changes , 
closed-loop  res;  nsec  remained  os.'U'.n'  i illy  u.v:h  mqi  d. 
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for  the  variability  among  subjects;  there  is  mt  a great  d-  .1  to 
be  gained  by  adopting  the  "precis  Ly"  oi  ;iraal  scanning  stra  sgy. 
Thus , if  one  is  interested  in  pr<  iicting  perj  ince,  a rel  .Li- 
vely crude  model  of  scanning  may  be  adequate. 

2.3.3  Manual  Control  Models  for  Workload 

There  has  been  a substantial  concern  with  problems  of 
operator  workload  in  manual  control.  Much  of  this  effort  has 
been  motivated  by  research  in  aircraft  handlin':  qualities  and 
pilot  opinion.  However,  a significant  portion  of  the  research 
is  relevant  to  basic  questions  of  task  interference  and  t.h  • 
effects  of  attention-sharing  on  performance. 

Models  for  predicting  operator  performance  and  workload 
must  take  into  account  the  effects  of  interactions  among  several 
control  and/or  monitoring  tasks.  When  overt  visual  sampling 
is  involved, it  is  clear  that  there  will  be  degradation  that 
should  be  accounted  for  in  a model  for  the  operator.  The 
question  of  scanning  workload  has  been  treated  by  Senders  (1964), 
who  derived  an  expression  for  the  fraction  of  time  that  must 
be  devoted  to  a given  display  given  the  amplitude  and  bandwidth 
of  the  displayed  signal  and  the  permissable  rr.s  reading  err  >r. 
With  this  relationship,  the  total  workload  pi : cod  on  an  id<  1 
observer  by  a given  set  of  displays  can  be  co:  ut.cd.  The 

extension  of  these  ideas  to  problems  involvin'  closed-loop 
control  has  been  carried  out  by  McRuer  ot.  al.  (19GB)  end  th  : 
colleagues  (see  Section  2 . 3.  2,  al  ).  This  pr<  les  > chan  is 
for  treating  scanning  workload  in  the  context  of.  describing 
tune tion  models.  Th<  • 1 -‘  ; 1 1 d wbacks  of  . i «ch  h 

been  discussed  previously. 
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Task  interference  and/or  attention  sharing  are  not  limited 
to  situations  involving  overt  scannin  For  example,  Brainerd 
et  al.  (linil?)  found  that  react  Lon  tirn  Increased  on  t !.  -ag< 

as  the  number  of  stimulus-response  channels  increased.  Since  in 
his  experiments  all  the  visual  stimuli  could  be  monitored  simul- 
taneously and  all  motor  responses  executed  simultaneously,  no 
overt  sampling  was  required.  There  was,  therefore,  interference 
at  the  central  processing  level  as  the  number  of  tasks  was 
increased . 


Numerous  studies  of  tracking  in  multi-variable  situations 
(e.g.,  Chernikoff  et  al.,  1959;  Todosiev,  et  al.,  1965;  Levison 
and  Elkind,  1967),  generally  confirm  that  performance  in  a given 
axis  is  degraded  when  an  additional  task  is  added.  The  amount 
of  degradation  will  be  very  dependent  on  the  experimental 
situation  and  in  some  cases  may  not  even  be  signif i cant  (e.g., 
Todosiev  et  al,  1965)  . In  addit  ion,  changes  in  d sscril  Lng  fund 
have  been  observed,  though  these  changes, too,  depend  on  the  part  i 
cular  characteristics  of  the  experiment.  A third  effect  is  that 
as  displays  become  less  integrated  and/or  more  separated,  per- 
formance degradations  increase?,  and  are  worst  when  overt  sc.mnin<; 
is  necessary. 

Although  there  is  considerable  concern  with  workload  i m 
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number  of  loops  increase,  this  approach  is  likely  to  run  into 
additional  problems  unless  methods  for  considering  interference 
in  this  context  are  developed. 

Levison,  et  al . (1971,  Abstract  Ho.  17)  have  proposed  a 
model  for  task  interference  and  attentional  workload  that  is 
appropriate  to  the  optimal  control  model.  The  workload  model , in  con- 
junction with  the  OCM,  allows  one  to  predict  how  the  operator  will 

allocate  his  available  attentional  capacity  among  tasks.  It 
also  allows  one  to  determine  the  relative  attentional  workload 
for  a given  task.  The  model  is  intended  to  apply  to  situations 
(such  as  continuous  manual  control)  in  which  the  human  is  required 
to  operate  as  a continuous  processor  of  information.  Very  briefly, 
the  model  for  interference  assumes  that  as  the  human  devotes  less 
attention  to  a given  displayed  variable, the  noise  on  that 
variable  increases  proportionally.  Thus,  if  P is  the  observa- 
tion noise/signal  ratio  when  "full  attention"  is  devoted  to  a 
task,  then  when  the  subject  is  forced  to  pay  less  than  full 
attention  to  the  i^1  task,  the  effective  noise/signal  ratio  is 


x 


where  f^  is  the  fraction  of  attention , devoted  to  the  1 n task 
0 £ fj,  < 1/  ar>d  JA  f^  = 1.  Clearly,  because  of  the  increase  in 
noise,  attention  sharing  will  degrade  performance. 

The  above  equation  can  be  used  to  help  predict  the  effects 
of  a given  allocation  of  attention  on  performance.  In  particular, 
the  equation  serves  to  determine  the  observation  noise  covariances 
associated  with  that  allocation  (specific  values  for  f.  and  P ) . 
Then  the  optimal-control  model  is  used  to  predict  pilot  behavior 
and  overall  system  performance.  If  it  is  not  known  beforehand  how 
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the  operator  will  allocate  his  attention,  mode!  solutions  are 
obtained  to  predict  the  optimum  allocation  of  attention  alone 
with  other  measures  of  interest.  In  keeping  with  the  fundamental 
optimality  hypothesis, the  optimum  allocation  of  attention  may  be 
taken  as  a prediction  of  the  operator's  allocation. 

Building  on  the  model  for  attention,  a "workload  index"  can 
be  defined  as  the  fraction  of  attention  required  to  achieve  a 
specified  criterion  level  of  performance  on  the  control  task. 

Thus , 

workload  index  = f+- 

Lc 

where  f^.  is  the  minimum  fraction  of  attention  for  which  performance 
can  be  maintained  within  the  criterion  level.  In  order  to  predict 
the  workload  index,  it  is  necessary  to  specify  a relevant  per- 
formance measure,  the  required  level  of  performance,  and  the 
"reference"  noise/signal  ratio  P . Levison  lets  Pq  correspond 
to  the  noise/signal  ratio  obtained  in  a standardized  laboratory 
situation  in  which  the  operator  is  motivated  to  minimize  his 
tracking  errors.  Although  this  value  does  not  necessarily 
correspond  to  "full  capacity,"  it  does  appear  to  correspond  to 
a high  workload  condition,  and  operation  at  this  level  for  any 
prolonged  time  would  undoubtedly  be  unacceptable . Of  course, 
when  one  is  interested  primarily  in  the  relative  change  in  work- 
load requirements  from  one  situation  to  the  next,  the  value  for 

P is  not  too  critical, 
o 

Levison's  model  of  attention-sharing  has  been  validated  foi 
multivariable  manual  control  tasks  (Lovison,  ok  al.,  1971)  and  for 
two  decision  tasks  (Levi  son  and  Tanner,  1971).  For  t he  control 
task,  whore  detailed  mcasurem.  nit  s of  hur.  response  were  available. 


i 

I 
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it  was  found  that  operator  remnant  and  describing  function 
changes  and  system  error  scores  were  affected  by  multipJ e-task 
requirements  in  the  manner  predicted  by  the  model.  It  was  also 
found  that  overall  decrements  in  performance  were  predicted 
better  than  individual  task  measures.  This  result  appears  to 
be  a consequence  of  the  insensitivity  of  total  performance  to 
reasonable  deviations  from  optimality  in  the  allocation  of  atten- 
tion among  individual  tasks. 


A limitation  of  Levison's  approach  is  its  inability  to 
predict  absolute  (rather  than  relative)  workload.  As  noted  by 
Wewerinke  (1974)  the  above  model  deals  with  selective  (rather 
than  intensive)  aspects  of  attention.  He  attempted  to  extend 
Levison's  model  to  include  the  intensive  aspects  of  attention 
and,  thereby,  predict  absolute  workload.  He  related  level  of 
arousal  in  a task  to  the  sensitivity  of  system  performance  with 
changes  in  attention  and  defined  an  "absolute"  workload  index  by 


VII  = ft  • s 
Lc 


v/here  S is  the  sensitivity  defined  as  the  fractional  change  in 
performance  for  a given  change  in  attention  evaluated  at  the 
criterion  level  of  attention.  This  predicted  WI  was  compared  with 
subjective  pilot  ratings  of  task  difficulty  for  six  control  tasks 
of  varying  difficulty;  there  was  a strong  linear  correlation 
(r  - .99)  between  the  two. 


Finally,  we  should  note  that  Levison's  modal  applies  to 
the  prediction  of  scanning  workload  as  well.  For  scanning,  the 
fractions  of  attention  sum  to  less  than  unity  because  of  the 
loss  of  information  during  large  eye  movements.  Once  this  lost 
time  is  accounted  for, one  can  determine  the  optimal  dwell 
fractions.  However,  scanning  workload  is  not  predicted  by 


I 
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adding  dwell  fractions  as  in  Senders'  models;  rather,  it  is 
determined  by  computing  , subject  to  the  modi fied 
constraints . 


2.3.4  Decision-Making  Models  in  Manual  Control 

In  manual  control, the  main  attempts  at  modeling  decision- 
making performance  have  been  directed  at  problems  of  failure 
detection  and  identification,  unless  one  considers  scanning  as 
a decision-making  process.  (See  Young  (1969)  for  a fine  review 
of  the  early  work  in  this  area) . Most  of  the  effort  concerns 
behavior  or  performance  following  an  abrupt  change  in  system 
dynamics,  as  might  occur,  e.g.,  in  a failure  of  an  aircraft 
stability  augmentation  scheme  (Elkind  and  Miller,  1967, 

Abstract  No.  18;  phatak  and  Bckey,  1969,  A.bs  tract  No.  9; 

Niemala  and  Krendel,  1974).  It  is  generally  agreed  that  the 
adaptive  response  of  human  controllers  in  failure  situations  may 
be  considered  to  comprise  the  following  basic  phases 
(Elkind  and  Miller,  1967;  Phatak  and  Bekey,  1969;  Phatak  and 
Kleinman,  1972): 


(1)  Control  of  pre-failure  plant  dynamics 

(2)  Detection  of  a change  or  failure  in  plant  dynamics 

(3)  Identification  of  now  dynamics 

(4)  Modification  and  optimization  of  control  strategy 
as  appropriate  to  new  dynamics 


The  first  and  fourth  phases  of  this  adaptive 
by  traditional  manual  control  models , whereas 
identification  are  addressed  by  decision-mak 


response  are  modeled 
detection  and 
ng  models. 
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Two  fundamentally  different  approaches  to  modeling  the  process 
of  detecting  a failure  are  evident  in  the  classical  manual  control 
literature.  One  approach  relies  on  the  concept  of  detecting 
patterns  in  the  observed  signals  while  the  other  employs  the 
concept  of  an  internal  model.  Frequently,  the  pattern  recognition 
approach  has  reduced  to  defining  decision  boundaries  in  the  phase- 
plane  (error  — error  rate  space) : crossing  appropriate  boundaries 

in  this  space  is  taken  as  an  indication  of  a change  in  pattern 
warranting  a decision  that  a change  in  dynamics  (or  a failure) 
has  occurred.  Perhaps  the  most  significant  exemplar  of  this 
approach  is  provided  by  Phatak  and  Bekey  (1969),  who  performed 
a detailed  and  interesting  analysis  of  adaptation  to  stability 
augmentation  failures  in  a VTOL  aircraft. 


The  alternative  approach  to  failure  detection  is  based 
on  the  notion  that  detection  occurs  when  an  unexpected  response 
is  observed  and  when  the  operator  is  conditioned  to  accept  the 
possibility  or  probability  of  a failure.  This  approach  depends 
on  an  internal  model  for  the  system  to  generate  the  expected 
response.  Several  schemes  of  this  type  have  been  proposed 
(Young  and  Stark,  1965;  Knoop  and  Fu,  1964;  Miller,  1965;  Elkind 
and  Miller,  1967).  The  work  of  Elkind  and  Miller  (1967)  seems 
to  be  the  most  sophisticated  and  detailed  of  these  efforts.  They 
postulate  that  the  operator  continuously  monitors  the  change  in 
the  system  error  rate  that  results  from  his  control  inputs,  and 
compares  this  change  with  the  predicted  change  that  he  expects 
on  the  basis  of  his  model  of  the  system  dynamics.  If  the 
difference  be  tween  the  observed  and  expected  changes  in  error 
rates,  relative  to  the  standard  deviation  of  the  distribution 
of  expected  differences,  exceeds  a criterion  value,  the  model 
reports  that  a change  in  system  dynamics  has  occurred. 


S') 
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Models  for  identification  of  the  post-failure  dynamics 
for  the  above  approaches  follow  from  the  detection  models.  A 
common  theme,  however,  is  that  the  operator  is  familiar  with 
(i.e.,  has  a model  of)  the  post-failure  dynamics.  Thus,  in  the 
Phatak  and  Bekey  model , a supervisory  algorithm  has  stored 
strategies,  as  determined  from  quasilinear  manual  control  models, 
appropriate  to  each  of  three  possible  failures.  The  model 
goes  through  a sequential  process  based  on  observation  of  error 
and  error-rate.  In  particular,  once  a failure  has  been  detected  (i.e., 
when  the  phase  trajectory  leaves  the  "unfailed  region"  of  phase 
space) , the  strategy  for  one  set  of  post-failure  dynamics  is  tried. 
Based  on  changes  in  the  sign  of  error-rate  and  on  leaving  or 
entering  various  decision  regions  in  the  phase  plane,  the  model 
decides  whether  these  dynamics  are  appropriate  or  not.  It 
continues  this  process  until  it  stabilises  on  an  appropriate 
s trategy . 

The  Elkind  and  Miller  model  also  assumes  that  the  operator 
is  well- trained  and  familiar  with  all  possible  changes  in  the 
dynamics  that  might  occur.  Once  the  detection  portion  of  the 
model  reports  that  a change  in  system  dynamics  has  occurred,  the 
model  then  enters  a Bayesian  analysis  phase  in  which  the  controller 
compares  the  observed  change  in  error  rate  with  that  expected  under 
alternative  hypotheses  concerning  the  system  change  and  selects 
that  set  of  alternative  dynamics  that  best  accounts  for  the 
observed  results.  Usually,  only  one  candidate  will  match 
adequately,  but  if  not,  identification  is  postponed  and  additional 
observation  are  accumulated  until  an  unequivocal  choice  can  be 
made.  Tn  addition  to  the  distinction  be'...  an  what  is  processed 
to  detect  a failure,  two  other  features  of  this  identification 
scheme  distinguish  it  from  that  of  the  Phatak  and  Bekey  model: 


Report  No.  34 4 6 


Bolt  Be ran ok  and  Newman  Inc. 


(1)  the  use  of  statistical  decision  theory  (hypothesis  testing) 
for  deciding  among  alternative  dynamics,  and  (2)  the  use  of  a 
parallel  rather  than  a sequential  selection  process. 

The  models  of  Phatak  and  Bekey  and  Elkind  and  Miller  have 
each  been  compared  with  experimental  data  in  a limited  number  of 
situations.  These  comparisons  suggest  at  least  good  qualitative 
agreement  and, in  the  latter  case,  some  quantitative  agreement  as 
well.  However,  the  models  seem  to  be  limited  in  their  potential 
for  application  to  new  problems.  For  example,  the  idea  of  error 
pattern  recognition  is  general  but  the  mechanization  of  decision 
surfaces  is  clearly  dependent  on  the  specific  dynamics.  Phatak 
and  Bekey  do  not  provide  any  suggestions  or  theory  as  to  how  to 
choose  such  regions  a priori  (i.e.,  before  an  experiment).  With 
respect  to  the  Elkind  and  Miller  model,  their  specific  implemen- 
tation would  appear  to  have  conceptual,  if  not  practical, 
difficulties  with  detection  of  failures  in  higher  order  or 
multivariable  plants  (Young,  1969)  . 

Developments  in  modern  control  and  estimation  theory  have 
paved  the  way  for  extension  of  the  above  concepts  to  decision- 
making models  of  potentially  greater  generality.  These  develop- 
ments are  reflected  in  the  information  processing  structure  of 
the  OCM  and  the  resulting  decision-making  models  may,  therefore, 
be  viewed  as  extensions  of  the  OCM.  However,  even  within  this 
common  framework  there  is  considerable  room  for  diversity.  To 
see  this,  let  us  briefly  review  the  salient  features  of  that 
information  processing  structure. 

Recall  that  the  OCM  contains  an  optimal  estimator  as  part  of 
its  perceptual,  processor.  The  estimator  processes  the  available 
observations  to  generate  a minimum  variance  estimate  of  the  system 
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state,  x(t),  an 
that  estimate;. 

.is  of  .interest, 
system  is  used 

observation.  The  predicted  state  estimate  is  updated  or 
corrected  using  the  observation  "residual,”  v(t),  which  is 
defined  as  the  difference  between  the  actual  observation  and 
the  expected  observation.  When  the  estimation  process  is 
optimal,  which  requires  that  the  internal  model  be  identical  to 
the  true  system  model,  the  residuals  are  a white  noise  process 
(with  covariance  V (t)  equal  to  the  observation  noise  covariance); 
in  other  words,  till  information  about  the  state  of  the  system  has 
been  extracted  from  the  measurements. 

It  is  apparent  from  the  above  that  the  optimal  estimator 
generates  variables  appropriate  for  either  a pattern  recognition 
approach  (x(t))  or  an  internal  model  approach  (v  (t) ) to  detection. 
If  just  this  information  is  used,  one  can  develop  descriptive 
models  of  detection  or  decision-making.  For  example,  a purely 
descriptive  model  based  on  the  state  estimate  could  involve 
decision  regions  in  state  space  just  as  in  the  pattern  recogni- 
tion models.  Alternatively,  a descriptive  detection  model  could 
be  based  on  a test  on  the  residuals  analogous  to  Elkind  and 
Miller's  scheme  or  on  a test  of  the  "whiteness"  of  the  process. 
Note  that  the  latter  schemes  are  not  tied  to  any  particular  set 
of  dynamics. 

For  prescriptive  in.  :o Is  of  decision-making,  optimal 

a 

decisions  must  be  generated , and  one  cannot  rely  solely  on  x(t) 
or  v(t).  However,  the  necessary  information  for  making  such 
decisions  is  provided  by  the  optimal  filter  in  cither  (x(t), 

>'.(1))  or  in  (\>(t),  V^.(t))  . Whether  one  uses  (x,  ) or  (v,V  ) 
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The  manner  in  which  this  estimate  is  generated" 
too.  In  particular,  an  internal  model  of  the 
to  predict  an  estimate  of  the  state  find  of  the 
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as  an  input  to  the  decision  mechanism  depends  to  some  degree  on 
the  particular  problem  and  to  some  degree  on  one's  view  of  the 
operator's  perceptual  process.  From  an  analytical  standpoint, 
it  would  appear  that  the  use  of  the  residuals  is  preferable  for 
several  reasons  (Gai.  and  Curry,  1976).  In  the  two  applications 
of  these  techniques  to  be  discussed  shortly,  Levison  and  Tanner 
(1971)  use  the  state  estimate  and  Gai  and  Curry  (197G)  the 
residual  representation. 

Many  of  the  above  ideas  have  been  discussed  by  Phatak 
and  Kleinman  (1972) . They  also  showed  how  one  could  employ 
"multiple"  internal  models  in  the  OCM  structure  to  perform 
simultaneous  detection  and  identification.  Their  paper  was 
theoretical  and  did  not  contain  validation  data.  However, 
two  studies  of  detection  that  have  tested  some  of  these  ideas 
are  now  discussed. 

Levison  and  Tanner  (1971,  Abstract  No.  19)  studied  the 
problem  of  how  well  subjects  could  determine  whether  a signal 
embedded  in  added  noise  was  within  specified  tolerances.  Their 
experiments  were  a visual  analog  of  classical  signal  detection 
experiments  except  that "signal-present"  corresponded  to  the 
situation  of  the  signal  being  within  tolerance.  From  a practical 
standpoint,  the  decision  is  analogous  to  the  problem  of  deciding 
whether  conditions  warrant  a given  action;  e.g.,  deciding  whether 
one  is  in  the  approach  window  on  landing  or  deciding  whether 
tracking  errors  are  small  enough  for  weapons  release.  The 
problem  differs  from  those  we  have  discussed  thus  far  in  that 
it  is  not  concerned  with  an  abrupt  change  in  system  dynamics  as 
in  a sudden  failure. 
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Levi son  modeled  this  situation  using  the  OCM  information 
processor  and  assuming  that  the  operator  is  an  optimal  decision 
maker  in  the  sense  of  maximizing  expected  utility.  For  equal 
penalties  on  missed  detections  and  false  alarms,  this  rule 
reduces  to  one  of  minimizing  expected  decision  error.  The 
decision  rule  is,  simply,  a likelihood  ratio  test/ which  Levison 
implemented  using  the  pair  (x(t),  E_(t)).  This  pair  is  appropriate 
for  this  problem  because  the  decision  depends  wholly  on  the  system 
sta  te . 


Model  predictions  compared  favorably  with  experimental 
conditions  for  a variety  of  cases  involving  different  signal/ 
noise  ratios  and  different  noise  bandwidth.  In  addition,  Levison 
tested  the  model  for  the  case  of  two  concurrent  decision  tasks 
and  a concurrent  decision  and  manual  control  task.  This  tested 
the  task  interference  model  for  the  OCM  (Levison,  et  al.,  1971) 
as  well  as  the  decision  making  model.  Good  results  were  obtained 
for  the  two  decision  tasks  but  not  for  the  combined  control/ 
decision  task.  Levison  suggests  a number  of  methodological 
explanations  for  the  lack  of  agreement  in  the  latter  case,  but 
this  remains  an  area  for  further  investigation. 

Gai  and  Curry  (1975,  1976 , Abstract  No.  20)  have  used  the 
OCM  information  processing  structure  to  analyze  failure  detection 
in  a simple  laboratory  task  and  Ln  an  experiment  simulating  pilot 
monitoring  of  an  automatic  approach.  In  both  cases  the  "failure" 
did  not  correspond  to  a change  in  system  dynamics  but,  rather, 
was  analogous  to  an  .instrument  failure.  To  be  more  specific,  a 
step  or  ramp  was  added  to  the  observed  signal  at  a random  time. 
This  produced  a non-zero  mean  value  for  the  signal  and  for  the 
residual;  failure  detection  consisted  of  testing  an  hypothesis 
concerning  the  mean  of  the  distribution  of  tlv  residuals. 
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Gal  and  Curry  used  sequent ial  analysis  to  perform  the 
hypothesis  tost.  By  summing  (or  integrating)  the  residuals, 
a likelihood  ratio  can  be  calculated  and  used  to  arrive  at  the 
decision.  Gai  and  Curry  modified  classical  sequential  analysis 
to  account  for  the  fact  that  a failure  detection  problem  is 
characterized  by  a transition  from  one  mode  of  operation  to 
another  at  a random  time  and  the  classical  analysis  is  based 
on  the  assumption  that  the  same  mode  of  operation  exists  during 
the  entire  observation  interval. 

They  reported  good  agreement  between  predicted  and  observed 
detection  times  for  both  the  simple  and  more  realistic  situations. 

In  the  latter  case,  the  model  is  used  in  a multi-instrument 
monitoring  task  and  accounts  for  attention  sharing  (via  Levison's 
model)  and  cross-chccking  of  instruments  to  confirm  a failure. 

A significant  result  of  the  experiments  is  that  the  property  of 
integration  of  the  residuals  appeared  to  be  confirmed  for  both 
step  and  ramp  type  failures. 

The  studies  of  Levison  and  Tanner  and  Gai  and  Curry  suggest 
significant  potential  for  the  use  of  the  modern  estimation 
techniques  in  modeling  decision-making.  However,  they  remain 
limited  in  scope;  for  example,  techniques  for  identifying  a 
failure  such  as  those  proposed  by  Phatak  and  Kleinman  have 
yet  to  bo  tested.  We  believe  the  most  significant  aspect  of  this 
work  is  that  it  provides  verification  of  the  information  processing 
structure  of  the  optimal  control  model  in  situations  not  involving 
closed-loop  control.  When  these  results  are  added  to  the  implicit 
validation  provided  by  the  tracking  data,  one  has  a strong  case 
for  this  type  of  modeling  of  human  information  processing.  A 
final  point  worthy  of  note  is  the  analogy  between  this  approach 
and  that  of  classical  signal  detection  theory  (Green  and  Swots,  19G6) 
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Here,  as  In  the  classical  theory,  tho  perceptual  process  is 
corrupted  by  an  internal  (observation)  noise,  and,  moreover,  it 
is  separated  from  the  decision  criteria.  It  would  appear  that 
this  analogy  would  be  fruitful  to  explore. 

2.3.5  Summa ry 

Manual  control  models  are  undoubtedly  unique  among  human 
performance  models  in  the  degree  of  their  quantitative  description 
and  the  extent  to  which  they  have  been  validated  and  applied.  The 
models  have  been  employed  to  analyze  new  flight  control  and  display 
problems,  to  determine  the  effects  of  stressors  on  performance,  and 
to  assist  in  experimental  and  simulation  planning.  It  is  clear  that 
the  problem  of  modeling  single-input,  single  output,  time-invariant 
systems  is  essentially  solved.  The  corresponding  multi-loop 
modeling  problem,  though  not  completely  solved,  is  in  releitivoly 
good  shape  too.  On  the  other  hand,  models  for  time-varying 
systems,  non-linear  systems  and  for  supervisory  control  are, 
comparatively,  only  at  an  early  stage  of  development.  This  is 
true,  too,  of  the  problems  associated  with  predicting  the  effects 
of  stress.  Moreover,  all  the  models  arc  designed  to  predict  the 
performance  of  well  trained  subjects,  usually  drawn  from  a group 
(pilots)  that  have  already  undergone  a pre-selection  process.  At 
present,  very  little  of  a quantitative  nature  can  be  said  concerning 
performance  during  training. 

With  respect  to  the  basic  controller  models  that  have  been 
developed,  any  or  all  of  them  could  prove  useful  in  modeling 
aspects  of  the  command  and  control  situation  that  involve  con- 
tinuous control.  However,  the  real  challenge  will  be  to  sec  if 
the  techniques  developed  for  manual  control  can  produce  similar 
success  in  modeling  monitoring  and  decision-making  tasks,  which 
usually  require  infrequent  control  action,  that  are  typical  of 
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large  portions  of  the  problem.  Results  to  date  suggest  that 
the  information  processing  structure  of  the  optimal  control 
model  has  significant  potential  in  this  area  but  that  much  more 
work  is  required  before  this  potential  can  be  realized. 
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2.4  Models  of  Human  Information  Processing 

The  idea  of  dividing  mental  operations  into  component  parts 
on  the  basis  of  latency  measures  has  been  employed  off  and  on 
since  the  time  of  Condors  (1869).  I [ova  or,  it  was  Broadbent, 

(1958)  in  his  seminal  book  on  human  information  processing,  who 
first  popularized  the  idea  that  it  is  possible  to  model  component 
processes  in  human  information-processing  systems  as  relatively 
independent  entities.  Building  on  Shannon's  notions  of  information 
as  a measurable  quantity,  Broadbent  addressed  the  possibility  of 
tracing  the  flow  of  information  through  the  human  sensory-motor 
system  one  stage  at  a time. 

Parallel  to  these  developments,  a new  field  of  mathematical 
psychology  was  evolving  from  purely  statistical  analyses  of 
experiments  and  psychometric  measurements  to  encompass  the  proposal 
of  formal  statistical  models  for  learning  (Rush  and  Hosteller,  1955), 
and  statistical  models  for  sensory  processes  (Tanner  and  Swets, 

1954)  . 

The  work  on  models  for  learning  slowly  coalesced  with 
information-processing  theory  to  place  emphasis  on  memory  as  .a 
process  rather  than  on  learning  per  so.  The  resources  of 
mathematical  psychologists  began  to  diffuse  toward  models  for  a 
variety  oT  information-processing  operations  ranging  from  percep- 
tion through  memory  and  response. 

During  the  last  twenty  years,  a healthy  interplay  has 
developed  betw  • ti  mathematical  mo. ! ■ 1 1 or  s , who  began  col  loci  i nq 
data  to  evaluate  their  theories,  a .d  experimental  psychologists, 
who  sought  quantitative  models  to  explain  their  data. 
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A third  thrust  in  the;  development  of  information  processing 
models  was  provided  by  the  advent  of  the  computer  as  a tool  for 
simulation.  For  those  interested  in  psychology,  this  impact  was 
felt  in  the  formulation  of  computer  simulation  models  for  problem 
solving  (Newell,  Shaw  and  Simon,  1958)  and  other  information- 
processing activities  such  as  memory  (Feigenbaum,  1961).  Like  the 
stage- modelling  approach,  these  efforts  reflect  the  convergence  of 
several  different  philosophical,  methodological,  and  technical 
threads . 

The  theoretical  position  of  investigators  in  this  area  is 
that  human  problem  solving  can  be  understood  and,  therefore, 
modelled  through  characterization  of  three  major  dimensions: 

(1)  the  task  environment,  including  the  problem  itself  and  the 
means  (e.g.,  rules)  available  for  solution;  (2)  the  problem  space 
employed  by  the  subject  to  represent  the  task  environment  and  an 
evolving  solution;  and  (3)  the  "program"  developed  to  achieve 
solution.  The  procedures  used  to  characterize  these  determinants 
involve  intensive  prior  analysis  of  the  problem  to  be  solved  and 
equally  intensive  collection  of  subject  protocols  during  solution. 
These  techniques  have  included  such  problems  as  (1)  choices  among 
alternative  moves  in  the  game  of  chess;  (2)  cryptar i tljmet  i c 
problems,  in  which  the  problem  solver  must  discover  a code  that 
substitutes  a unique  set  of  numeric  characters  for  a given  set  of 
alphabetic  characters;  and  (3)  the  development  of  proofs  for 
problems  in  symbolic  logic  via  formal  application  of  appropriate 
rules . 

The  major  goal  of  this  research  is  to  develop,  from  the 
protocols  of  subjects  and  the  problem-behavior  graphs  to  which 
these  give  rise  during  subsequent  analysis,  a computer  program 
that  captures  and  reproduces  both  process  and  solution.  Tin's  goal 
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has  boon  approximated  in  the  formulation  of  the  GPS  (General 
Problem  Solver)  program,  which  appears  to  simulate  quite  well  the 
performance  of  skilled  subjects  on  a variety  of  logical  problems. 

As  a result,  it  represents,  in  the  view  of  its  authors,  not  only  a 
valuable  technical  achievement  but  also  a convincing  and  testable 
theory  of  human  problem  solving. 

Today,  the  distinction  among  the  empiricists,  the  mathematical 
modellers,  and  the  computer  modellers  has  blurred  as  the  results 
of  experiments  tend  to  be  expressed  in  models,  as  the  complexity 
of  the  models  calls  for  computer  implementation,  and  as  the  com- 
puter modellers  look  beyond  subjective  protocols  to  experimental 
results  as  the  basis  for  cognitive  simulations. 

2.4.1  Integrative  Information  Processing  Models 

In  the  course  of  our  explorations  of  the  problem  of  large- 
scale  simulation  models  it  has  become  clear  that  one  of  the  major 
difficulties  lies  with  the  inability  to  generalize  from  predictions 
generated  on  the  basis  of  one  scenario  to  others  involving  changed 
levels  for  particular  system  variables  such  as  number  of  operators, 
communications  bandwidth  available  or  number  of  objects  being 
controlled.  One  solution  to  this  problem  may  be  to  consider  ir> 
advance  the  range  of  system  conditions  for  which  generality  is 
sought,  to  carefully  modularize  the  model  so  that  those  systems 
variables  can  bo  accommodated  and  then  to  proceed  to  more  detail  d 
design.  Another  soldi  ion,  or  | • •vhaps  on  - to  be  applied  concurrently , 
is  to  build  a very  general  model  of  human  performance  at  a level 
of  detail  that  makes  very  few  assumptions  about  the  specifics  of 
system  implementation  and  simply  to  bui 1 d the  system  model  around 
this  general  model  of  ‘he  human  operator.  It  if:  to  this  end  that 
wg h ive  revi ewe  1 the  mode It ing  work  in  human  information  processing 
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because  that  is  clearly  the  goal  of  those  involved  in  much  human 
information  processing  research. 

Perhaps  the  major  conclusion  we  have  reached  in  this  regard 
is  that  the  state-of-the-art  does  not  provide  the  kind  of 
integrative  models  necessary  to  produce  such  a representation  of 
human  performance  that  could  be  tailored  to  this  class  of 
application.  There  are  two  major  deficiencies  that  need 
to  be  resolved  before  this  kind  of  capability  would  be  feasible: 
we  need  a better  understanding  of  how  to  relate  task  requirements 
to  elemental  human  performance  capacities  and  limitations, and  we 
need  models  of  performance  that  are  more  comprehensive  and  less 
compartmentalized. 

The  problem  of  translating  task  demands  into  components  that 
can  be  related  to  human  performance  models  is  essentially  a 
taxonomic  problem.  We  need  a methodology  for  analyzing  tasks  that 
abstracts  the  essential  human  performance  requirements  in  a form 
commensurate  with  available  model  structures.  While  this  is 
possible  on  the  basis  of  intuition  and  art  if  we  stick  to 
relatively  simple  tasks  such  as  reading  meters  or  moving  switches 
or  controls,  more  complicated  tasks  typical  of  integrative  behavior 
do  not  yield  easily  to  such  analysis.  Fleishman  (Theologus  and 
Fleishman,  1971)  has  proposed  an  approach  to  defining  a task 
taxonomy  that  utilizes  factor  analysis  to  define  human  ability 
dimensions  that  could  then  be  associated  with  tasks  to  be  analyzed 
through  the  systematic  collection  of  judges' ratings  of  the 
abilities  important  to  cacli  task.  The  method  is  .ingenious,  but 
would  require  extensive  empirical  research  to  derive  the:  abilities 
and  validate  judgment  scales  for  a broad  enough  sample  of  abilities 
to  be  useful  for  taxonomic  purposes  . Furthermore,  the  t axonomy  that  would 
result  would  not  bo  compatible  with  predictive  modelling  of  the 
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typc>  under  consideration  in  this  report  since  it  is  ability- 
oriented  rather  than  process-oriented.  Teichner  (1974)  has 
defined  a task  taxonomy,  building  on  Posner's  informational  approach, 
that  is  promising  in  the  areas  where  there  is  much  empirical  data 
but  of  limited  usefulness  where  there  is  not.  Teichner  postulates 
a series  of  information  processing  stages  similar  to  those  of  most  othe 
formulations  and  argues  that  some  tasks  bypass  certain  stages 
altogether  while  others  cause  certain  stages  to  drop  out  as  a 
function  of  practice.  As  a general  approach,  this  seems  quite 
productive, but  applying  the  rules  and  deciding  when  to  eliminate 
mediating  stages  remains,  for  all  but  the  most  elemental  cases, 
an  empirical  discovery  process.  Thus,  the  articulation  between 
task  and  information  processing  model  remains  a black  art 
requiring  extensive  empirical  validation. 

With  respect  to  integrative  models,  the  literature  is  rife 
with  block  diagrams  of  relationships  among  the  postulated  stages 
of  human  information  processing  (e.g.,  Teichner,  1974;  Norman 
and  Ruramelhart , 1970;  Kahneman,  1973;  Horton,  1970).  Each  author 
usually  focuses  on  a detailed  analysis  of  one  or  more  subsets  of 
stages  that  are  then  shown  to  model  particular  laboratory  tasks 
effectively.  The  existing  models  emphasize  simple  verbal  memory 
tasks,  alphanumeric  identification  tasks,  or  the  perceptual 
aspects  of  information  processing, as  will  be  discussed  in  more 
detail  below.  To  our  knowledge, no  one  has  provided  the  specifi- 
cations that  would  convert  a comprehensive  block  diagram  into  a 
working  model  at  this  level  of  detail.  While  it  might  be  possible 
to  do  so  in  a specific  case,  it  is  our  opinion  that  the  current 
state  of  understanding  would  not  support  building  such  a model 
that  would  be  generally  applicable  at  this  intimate  a level  of 
dot ail. 
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An  a methodology,  the  approach  that  seems  i :ost  likely  to 
achieve  this  goal  is  the  modelling  effort  that  is  a part  of  the 
system  development  activities  of  the  Naval  Air  Development  Center 
under  the  direction  of  Cmdr.  Robert  Wherry , called  the  Human  Operator 
Simulator  (IIOS)  (set:  Abstract  No.  21).  HOS  is  to  provide  a computer 
simulation  of  human  performance  in  a goal-oriented,  task-processing 
environment  (Strieb,  1975).  It  is  used  in  conjunction  with  other 
programs  such  as  the  Human  Operator  Procedures  (HOPROC) , Human 
Operator  Data  Analyzer/Collator  (HODAC)  , and  the  HOPROC  Assembler/ 
Loader  (HAL)  to  provide  the  capability  for  full  simulation. 

Four  high  level  functions  are  defined: 

1.  The  decoder,  which  provides  the  translation  from  task 
requirements  to  human  performance  functions. 

2.  The  multiplexor , which  sets  priorities  for  the  order  in 
which  task  requirements  will  be  sequenced,  based  on  an 
index  of  criticality. 

3.  The  estimator,  which  is  the  sensing-remembering  core  of 
HOS  and  assigns  time  costs  to  the  elemental  human 
performance  operations  undertaken.  It  includes  a memory 
module  that  is  based  on  a strength  model  of  memory 
performance  having  a short-term  and  long-term  memory 
capability.  The  memory  module  is  the  major  probabilistic 
element  in  HOS.  The  estimator  also  includes  a model  of 
the  operator's  physical  and  biomechanical  operations  such 
as  eye  movements,  limb  movements,  etc. 

4.  The  banker,  which  accumulates  time  costs  and  serves  to 
interface  HOS  with  the  simulated  hardware  components  of 
the  system. 
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The  decoder  subsumes  the  taxonomic  aspects  of  the  modelling 
problem  we  have  discussed  above.  While  HOPROC  provides  a conven- 
ient means  for  entering  such  information,  as  far  as  wc  can  discern, 
it  introduces  no  advance  in  the  technology  of  converting  task 
requirements  into  performance  functions.  Using  the  NADC  system, 
this  activity  still  incorporates  task  analysis  and  the  intuition 
of  experts  knowledgeable  concerning  human  performance  to  derive 
the  detailed  specifications. 

The  essential  performance  modelling  activity  is  subsumed  in 
the  estimator  module.  It  is  here  that  the  elemental  information 
and  motion  processing  components  are  represented.  Detailed 
descriptions  of  the  models  that  have  been  vised  were  not  available 
at  the  time  of  this  report.  We  believe  that  the  ultimate  sviccess 
or  failure  of  IIOS  will  depend  on  the  success  with  which  these 
processes  are  represented,  which  is  again  the  same  issue  that  faces  all 
us  interested  in  modelling.  While  IIOS  does  provide  an  insightful 
framework  and  it  appears  that  at  this  level  some  modelling  has  been 
undertaken,  such  as  memory,  eye  movement  times,  and  limb  movement 
times,  the  predictions  of  information  extraction  time  are 
still  estimated  on  a case  by-case  basis,  where  they  cannot  bo  assume 
to  be  gencrically  constant.  IIOS  does  advance  the  state-of-the-art 
in  attempting  to  make  these  estimates  at  the  level  of  individual 
processing  activities,  however , instead  of  at  the  larger  task 
level  typical  of  most  network  models. 

An  important  feature  of  the  modelling  methodology  r;  presented 
by  IIOS  is  the  cumulative  incremental  approach  that  is  being  taken 
to  development,  and  validation.  Validation  of  IIOS  may  be  undertaken 
in  successively  larger  units.  Wo  arc  aware  of  validation  to 
predict  meter  reading  and  scanning  performance.  For  this  purpose, 
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the  models  required  for  this  subset  of  tasks  have  boer.  implemented 
and  tested  against  real  data.  When  these  models  have  been  tuned, 
the  underlying  human  performance  components  will  be  available  to 
predict  larger  units  of  behavior  of  which  instrument  reading  is  a 
subtask,  and  so  forth.  We  believe  that  this  is  an  admirably 
systematic,  though  time  consuming  way  to  proceed. 

A second  important  contribution  is  the  insightful  and  careful 
way  in  which  the  various  components  of  human  performance  are  being 
modularized  and  interrelated.  As  a result,  many  of  the  aspects  of 
human  performance  fall  out  as  a consequence  of  these  logical  rela- 
tionships, thereby  reducing  the  complexity  of  task  modelling 
requirements . 

As  will  be  discussed  in  a later  section,  one  cannot  evaluate 
the  success  of  a human  performance  modelling  effort  without 
understanding  in  considerable  detail  the  goals  to  which  it  is 
addressed.  Cmdr.  Wherry  argues  that  HOS  is  directed  at  system 
evaluation  down  to  the  level  of  detailed  workplace  layout  and 
control  and  display  design.  As  presently  conceived, it  is  certainly 
capable  of  that  level  of  detail.  The  question  of  interest  here  is 
whether  it  is  also  useful  for  evaluating  more  global  aspects  of 
complex  system  performance.  On  this  point  there  is,  as  yet,  no 
evidence.  The  modelling  of  more  complex  collections  of  tasks  in 
HOS  faces  the  same  problems  addressed  in  Section  2.2  on  network 
models.  It  uses  a bottom-up  approach,  and  its  success  at  complex 
levels  depends  on  the  modeller's  ability  to  structure  the  tasks  so 
that  the  main  effects  and  interactions  of  interest  are  observable 
and  measurable  and  arc  not  buried  in  the  implicit  structure  of  the 
model.  Supposing  that  global  system  performance  measures  wore 
the  vari  ables  of  interest  rather  than  workplace  layout  per  so , 
one  could  make  the  argument  that,  carrying  the  model  to  the  level 
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of  detail  they  propose  is  at  best  unnecessary  and  at  worst 
impractical,  in  the  sense  that  the  effort  required  in  time  and 
expense  to  produce  a system-level  validated  model  is  greater  than 
the  payoff  that  will  result.  It  seems  .likely  that  models  should 
be  developed  to  provide  answers  to  questions  of  interest  and  should 
only  represent  performance  to  the  level  of  detail  required  for 
answering  those  questions.  The  quest  for  the  ultimate  model  of 
human  performance,  a goal  that  one  might  read  into  the  HOS  efforts, 
may  be  unrealistic. 

2.4.2  Information  Processing  Model s of  _L i mited  Scope 

Given  these  reservations  about  the  development  of  a truly 
general  model  of  human  performance,  let  us  drop  back  a level  and 
ask  what  kinds  of  useful  process  models  of  more  limited  scope  are 
available.  We  do  not  pretend  to  have  conducted  an  exhaustive 
search  of  available  models,  and  undoubtedly  there  are  bits  and 
pieces  of  models  we  have  examined  and  chosen  not  to  describe 
that  might  at  a later  time  be  shown  to  be  useful.  The  information 
processing  models  for  which  we  have  provided  abstracts  in  the 
Appendix  are  representative  of  the  best  we  have  been  able  to 
identify  according  to  the  evaluation  criteria  such  as  relevance, 
complexity,  and  usefulness  in  the  topical  areas  identified  in  the 
introduction . 

2 . 4 . 2 . 1 Information  Measurement 

Some  of  the  first:  quantitative  efforts  to  model  human  per- 
formance were  derived  from  Shannon's  i n format  ion- 1 hoorotic  concepts. 
It  was  these  concepts  that  led  to  the  idea  that  an  operator's 
performance  could  be  viewed  as  a 1 imi ted-capa city, single-channel 
information  processing  system.  Posner  (1964)  distinguished 
i n format i on- conserving  tasks, in  which  there  is  a one-to-one  mapping 
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between  stimuli  and  required  responses,  from  information-condensing 
tasks  involving  a many-to-one  classification  and  from  information 
creation  tasks  in  which  there  is  a one-to-many  mapping  of  stimuli 
to  responses. 

In  those  information  conserving  situations  in  which  the 
stimulus  set  and  response  set  can  be  well-def ined,  either  in 
discrete  terms  or  in  terms  of  continuous  probability  distributions, 
there  is  a very  large  data  base  and  a sound  basis  for  making  pre- 
dictions of  the  time  required  for  processing.  While  this  is 
possible  in  many  practical  cases,  in  many  others  the  technical 
definition  of  the  effective  stimulus  and  the  basis  for  quantifying 
it  in  informational  terms  is  very  difficult  and  often  arbitrary 
(see  Abstract  No.  22) . 

As  a measurement  tool,  information  theory  offered  a unique 
way  of  combining  data  concerning  speed  of  performance  with  that 
of  accuracy  into  a single  metric,  and  it  was  because  of  this 
feature  that  it  gained  considerable  popularity  in  the  late  1950's 
and  early  '60's.  It  is  now  recognized  that  the  way  in  which 
this  relative  weighting  of  speed  and  accuracy  were  introduced 
represented  only  one  of  many  possibilities  and  that  it  may  not  always 
be  the  most  appropriate  one.  Further,  the  computation  of  rate 
of  information  transmission  requires  knowledge  of  the  full  matrix 
of  data  corresponding  to  the  joint  stimulus-response  probabilities 
on  a cell  by  cell  basis.  Where  prediction  of  performance  in  new 
situations  is  the  goal,  as  it  is  in  the  context  of  this  report, 
the  computation  requires  estimation  of  these  probabilities  in 
advance.  It  would  be  rare  to  have  such  advance  information 
without  having  accumulated  considerable  experience  with  the  system 
in  actual  operation.  A s a result,  the  predictive  usefulness  of 
the  information  metric  is  limited  to  cases  in  which  error  rates 
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can  be  assumed  to  be  negl  i'j  ible,  and  t ho  major  concern  is  with  the 
speed  of  .information  processing  per formance . 

Finally,  there  arc  many  factors  that  influence  the  rate  of 
information  processing  that  arc*  not  captured  by  the  information 
metric.  These  include  temporal  uncertainty,  stimulus-response 
compatibility,  practice  effects,  stimulus  intensity,  and  stimulus 
and  response  repetition  effects.  At  the  present  state  of  our 
understanding,  these  factors  must  be  dealt  with  as  a catalog, 
assigning  values  to  them  on  the  basis  of  previous  experimentation 
and  judgment.  V.’e  do  not  have  formal  models  with  which  to 
integrate  them  into  the  overall  information  flow  analysis. 

Because  of  these  limitations,  research  on  human  information 
processing  has  moved  toward  analysis  of  the  various  stages 
of  processing  that  have  been  postulated  for  the  human 
operator.  It  is  to  these  classes  of  models  that  we  turn  next, 
beginning  with  those  models  that  take  a decision-theoretic  view  of 
performance . 

2 . 4 . 2 . 2 Decision-Theory  Based  Models 

In  the  decision-theor ic  framework, the  environment  in  which 
decisions  are  made  is  cast  in  terms  of  (l)  a set  of  mutually 
exclusive  and  exhaustive  but  uncertain  states  of  the  world  to  which 
a priori  probabilities  are  assigned,  and  (2)  a set  of  alternative 
courses  of  action  or  decisions,  also  mutually  exclusive  and 
exhaustive,  that  may  be  taken.  The  observe  r accumulates  evidence 
over  time  concerning  the  probabilities  of  the  various  states  of 
the  wo  Id  and  evaluates  that  evidence  in  the  context  of  the  values 
and  costs  associated  with  combinations  of  each  state  of  the  wovld 
and  coarse  of  action. 
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When  one  is  interested  in  predicting  the  probability  of  a per- 
ceptual response  - target  detection,  recognition,  and  identification  - 
Signal  Detection  Theory  (Green  & Swots,  1966)  provides  the  most 
thoroughly  formulated  and  generally  applicable  model.  It  is 
essentially  a Rayesian  model  that  revises  the  a priori  probabil- 
ities of  alternative  states  of  the  world  based  on  human 
observation, and  this  information,  together  with  collateral  information 
concerning  the  values  and  costs  associated  with  action, leads  to  a 
prediction  of  the  choice  of  an  alternative  response  (see  Abstract  No. 
23).  The  elegance  and  power  of  the  model  derives  from  the  fact 
that  under  conditions  in  which  the  environmental  desorption  of  the 
signals  and  noise  in  which  they  are  embedded  can  be  expressed  in 
quantitative  terms,  it  is  possible  to  predict  the  performance  of 
an  ideal  statistical  observer  and  to  describe  human  observer 
performance  in  terms  of  deviations  from  this  ideal.  Even  under 
conditions  where  this  is  not  possible,  it  has  been  shown  empirically 
that  human  performance  frequently  relates  well  to  the  underlying 
parameters  of  the  model  and  that  it.  may  be  generalized  to  pattern  detection 
tasks  and  other  recognition  and  identification  tasks  for  which  an 
ideal  observer's  performance  would  be  incompletely  specified. 

Signal  Detection  Theory  is  usually  applied  to  discrete-trial 
tasks,  such  as  yes-no  detection  tasks  or  two-alternative  forced- 
choice  tasks.  To  apply  it  to  a surveillance  problem  in  which 
there  is  no  inherent  temporal  structure  requires  making  further 
assumptions  about  the  frequency  and  duration  of  observations. 

Egan  et  nl  (1961)  and  Luce  and  Green  (1970)  have  developed  gener- 
alizations that  can  handle  these  circumstances.  Lucas  (1967) 
formulated  a model  of:  ideal  performance  for  the  case  in  which  the 
time  period  of  surveillance  was  limited,  each  observation  incurred 
a cost,  and  the:  cumulative  observation  cost  war;  limited. 
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Most  applic.it  ions  ol'  detection  theory  have  been  as  a 
methodology  for  making  quantitative  measurements  of  sensory  and 
perceptual  capabilities  rather  than  .is  a predictive  model.  How- 
ever, there  are  several  applications  to  visual  inspection  tasks, 
and  the  model  has  been  shown  t.o  bo  applicable  to  vigilance 
behavior  (Swats,  1977). 


i 


Closely  related  to  the  signal  detection  perspective  is  the 
work  on  latency  models  for  human  discrimination,  identification, 
and  response.  Virtually  all  of  these  models  take  the  view  that 
evidence  accumulation  in  the  human  information  processing  system 
takes  time.  Broadbent  (1971),  adopting  the  position  put  forward 
by  Audley  and  Pike  (1965),  distinguishes  among  throe  alternative 
approaches:  (1)  accumulator  theories  with  perfect  memory,  in 

which  the  choice  of  response  is  based  on  a race  between  evidence 
accumulation  (in  discrete  units)  in  favor  of  each  response,  and  in 
which  the  response  corresponding  to  the  first  accumulator  to  reach 
a criterion  accumulator  count  is  the  one  chosen;  (2)  accumulator 
models  with  defective  memory,  in  which  the  proportion  of  discrete 
"observations’’  in  favor  of  a particular  response  must  exceed  a 
criterion,  but  only  evidence  through  a particular  "memory  window" 
is  accepted  and  older  observations  are  dropped;  and  (3)  random- 
walk  models  in  which  each  new  sample  or  observation  causes  a 
change  .in  the  relative  likelihood  of  alternative  responses  and 
thereby  causes  a shift  in  confidence  in  favor  of  eacli  response. 
Audley  and  Pike  (.1965)  and  Laming  (1965)  discuss  the  predictions 
of  these  models.  Edwards  (1965)  presents  the  most  completely 
formulated  random-walk  model,  but  the  mnther.m tics  are  only 
developed  for  the  two-alternative  case  and  it  is  not:  well 
val id a ted . 
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Vickers  (1970,  see  Abstract  No.  24)  presents  an  interesting 
variant  of  the  random-walk,  model  which  incorporates  the  concept 
that  sequential  samples  may  vary  in  the  magnitude  of  the  evidence 
they  contribute  to  a choice.  With  this  formulation,  he  is  able 
to  show  that  accumulator-models  and  the  standard  random-walk  model 
fall  out  as  special  cases.  Here  again,  Vickers  has  dealt  only 
with  the  two-alternative  case  and  has  validating  evidence  from  only  a 
limited  class  of  laboratory  tasks. 

If  one  wished  to  model  stimulus  discrimination  or  identifica- 
tion in  a practical  setting,  the  ideas  from  these  models  would  be 
useful,  but  it  would  require  further  model  development  and 
validation  to  ensure  their  applicability. 


A slightly  different  approach  to  stimulus  identification  has 
been  developed  by  several  researchers  who  have  been  more  interested 
in  recognition  and  identification  probability  under  tachistocopic 
conditions,  that  is,  where  the  information  is  only  briefly 
presented.  The  work  of  Rumelhart  (1970),  and  Rumelhart  and  Siplo 
(1974)  is  representative  of  the  most  completely  formulated  models 
for  this  purpose  ( see  Abstract  No . 25).  There  the  identification 
process  is  viewed  as  one  of  selective  feature  extr£iction,  with  the 
number  of  features  coded  in  the  display  time  available  leading  to 
a prediction  of  the  probability  of  pattern  identification.  The 
model  has  been  shown  to  be  successful  for  a remarkably  broad 
range  of  laboratory  conditions,  but  brief  exposures  are  reflective 
of  only  a very  small  segment  of  the  activity  undertaken  in  any 
practically  useful  context. 
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2.4.2. 3 Models  for  Short  Tei m Men o . y 

Perhaps  the  most  active  modelling  area  in  psycholog/  has  been 
concerned  with  representing  memory  performance.  Some  theorists 
believe  that  the  difference  between  short-term  and  long-term 
memory  characteristics  is  only  one  of  degree  (Melton,  1963). 
However,  most  specialists  now  agree  that  retention  and  loss  of 
information  from  short-term  memory  differs  qualitatively  from 
these  processes  in  long-term  memory.  Some  current  theorists  even 
propose  a three-stage  memory  system  including  an  intermediate  or 
working-memory  level  in  addition  to  the  two  usually  defined 
(Broadbent,  1970) . 

Early  formulations  of  memory  models  focussed  on  trying  to 
settle  the  issue  of  whether  an  all-or-none  process  provided  a 
better  representation  of  the  acquisition  of  information  in  memory 
as  contrasted  with  an  incremental  or  "strength"  theory  of  memory 
(see  Restle  and  Greeno,  1970,  for  a discussion  of  this  issue 
and  the  models  proposed) . 

A significant  advance  in  the  development  of  memory  modelling 
came  when  Atkinson  and  Shiffrin  (1968)  formulated  a model  that 
for  the  first  time  distinguished  explicitly  th  structur.  1 rep- 
resentation of  memory  stores  from  the  task-c!  'pendent  co  * rn  1 
strategies  used  for  encoding  information  in  tiv.n  stores  and 
interrelating  them.  (See  Abstract  No.  36.)  'fin  ; h|«  l also 

represents  one  of  the  most  comp  la  ' and,  t.l.  ;'•••  , , pot. a’  ielly 

useable  models  of  memory  perform;.  ••  • . 

Sound,  is,  Smith, and  To  i oh  nor  (1974)  ha\n  • i.  ...  i an  exten- 
sive col  lection  of  other  mod  I <•  • r>.  ao  y.  K«  1 1 i <->vo  t h-  y would 
suppo  rl  the  ,ir<(ii;'"T!t  that  none  of  ‘ ■ e:.  i : i in.-  ills  i re 
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forma la tod  .in  sufficient  detail  and  v; : t h sufficient  general  i ty 
to  translate  directly  into  a practical  context.  Kh.it  they 
represent  is  a set  of  concepts  that  might  ultimately  be  used  in 
models  for  memory  performance  in  a practical  context.  A further 
limitation  is  that  they  have  dealt  almost  entirely  with  memory 
for  verbal,  alphanumeric  symbols.  The  modelling  literature  for 
memory  of  pictorial  material  is  virtually  non-ex  istant , although 
some  interesting  experimental  studies  are  beginning  to  appear. 

2 . 4 . 2 . 4 Motor  Performance 

Most  of  the  interesting  work  modelling  human  motor  performance 
has  focused  on  closed-loop  tracking  control  as  discussed  extensively 
in  earlier  sections  of  this  review.  The  major  exception  to  this 
view  is  the  experimental  and  theoretical  work  related  to  what  has 
been  called  Fitts'  Law  (see  Abstract  No.  27).  This  work  provides 
a means  for  predicting  movement  time  given  the  amplitude  to  be 
moved  and  the  accuracy  tolerance  for  the  movement.  The  predictions 
have  been  validated  for  manual  movements  in  various  directions,  for 
foot  movements  (Drury,  1975),  and  for  head  movements.  Recently, 
Kvalseth  (1975)  has  generalized  the  results  to  take  account  of 
cases  in  which  the  targets  are  temporarily  obscured  un‘  il  the 
movement  is  actually  in  progress. 

For  simple  movements,  the  model  is  robust  and  effective.  As 
one  proceeds  to  more  complex  movement  s»  it  may  be  necessary  to  augment 
this  model  with  data  derived  from  pr-.  lot  o mined  t i me  systems  used 
in  industrial  cr.g  i neor  ing , a procodur-  that  op  - rates  with  tabled 
values  for  movement  Lime  foi  a variety  of  wel 1-defined,  oli  mental 
manual  movement  activities  (Kargor  and  Bayha,  1966) . Predetei  ined 
time  syst  ms  have  been  dove lo:  >d  in  manner  anal o<;ou.  to  the 
d • ‘ ; - bank  approach  to  t . i sk-  c.  n t > • i e 1 m r oi  i i e I i on:; . 
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2, 4. 2. 5 Intellectual  Tasks  and  Mental  Computat Lon 

As  wo  move  toward  higher  mental  processes,  models  become 
more  diffuse.  Consider  a question  as  seemingly  straightforward 
as  estimating  the  time  to  £idd  two  numbers  together.  If  we  restrict 
the  domain  to  single  digits  whose  sum  docs  not  exceed  18,  then  a 
counting  model  of  the  form  suggested  by  Croon  and  Parkmnn  (1972) 
is  very  suitable.  The  observer  takes  the  larger  of  the  two 
numbers  and  increments  it  serially  at  a rate  of  20  msec  per  integer 
until  the  sum  is  reached.  Seldom,  however,  would  such  a restricted 
domain  be  useful.  Thomas  (1963)  has  examined  addition  and  multi- 
plication and  has  shown  that  the  log  (a-tb+c)  where  a + b - c or 
a x b = c is  a reasonable  predictor  of  computation  time.  Again, 
however,  the  domain  is  rather  limited  and  many  assumptions  must  be 
made  to  extend  this  simple  model  even  to  encompass  routine  calcula- 
tion requirements.  Similar  efforts  have  been  directed  at  modelling 
the  time  for  simple  logical  inferences  but  the  results  are  specific 
to  the  circumstances  investigated  (set  inclusion  and  exclusion) 
and  are  unlikely  to  generalize. 

Considerable  .information  has  been  accumulated  concerning  human 
ability  to  make  simple  intuitive  statistical  inferences  (Peterson 
and  Boach , .1967,  (see  Abstract  No.  28).  These  data  focus  on  the 
accuracy  of  such  inferences  rather  than  the  time  for  such 
inferences.  Kahneman  and  Tvcrsky  (1972)  have  described  the  sorts 
of  heuristics  that  people  use  in  making  .inferences,  but  thus  far 
they  can  only  be  captured  in  gu a litat j ve  terms  (sec  Abstiact  No.  29). 
Both  of  these  sources  have  more  of  the  character  of  data  appropriate 
to  a da ta~ s tore  repres  uitation  rather  than  that  ol  a formal  model, 
but  they  might  be  useful  for  the  further  development  of  quant  itntive 
mod  • 1 : ; . 
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The*  one  area  in  which  considerable  success  has  been  achieved 
in  modelling  judgmen La  1 processes  is  that  related  to  probabilistic 
inference  under  uncertainty.  Again  drawing  from  the  decision- 
theoretic  framework,  the  basic  model  is  nothing  more  than  Bayes ' 
Theorem, and  it  provides  a normative  representation  for  how  a 
decision  maker  should  revise  his  estimates  in  the  light  of  new 
data  (see  Abstract  No.  30).  However,  some  of  the  experimental  work, 
such  as  Edwards,  Phillips,  Hays,  and  Goodman  (1968),  indicates  the 
systematic  ways  in  which  human  inference  is  degraded  when  data 
aggregation  is  required  and  could  be  used  as  a basis  for  a model . 
These  ideas  also  have  been  extended  to  model  the  impact  of 
unreliable  data,  a condition  frequently  encountered  in  the 
practical  world  (see  Abstract  No.  31).  It  should  also  be  made  clear 
what  they  do  not  do.  They  make  no  attempt  to  model  the  time 
required  for  judgment.  They  do  not  predict  the  specific  likelihood 
ratios  an  operator  would  assign  to  each  data  item;  such 
estimates  would  have  to  be  anticipated  and  provided  to  the  model. 

Perhaps  the  most  sophisticated  attempt  to  model  human  reason- 
ing processes  are  embodied  in  the  General  Problem  Solver  of  Newell 
and  Simon  (1963).  This  work  represented  the  peak  of  the  era  in 
which  protocols  of  the  activities  of  a human  subject  when  solving  a 
problem  were  described  in  a computer  program.  While  some  of  the 
approaches  taken  in  this  program  might,  provide  useful  concepts  tor 
the  ki  nd  of  model  1 ing.^  wo  are  concerned  with,  the-  goals  of  t his 
research  were  quite  different.  The  authors'  emphasis  was  on 
producing  a program  that  would  go  through  the  same  stops  as  a 
human  problem  solver.  It  placed  less  emphasis  on  predicting 
accurately  Lh  • frequency  with  which  th  ■ program  would  I i nd  a 
solution  in  eompat  i son  with  Ira  man  success  pmhabil  ity.  Furthei  , 
the  problems  deni t with  had  a sin  le  unique  solution,  a circumstance 
not  common  in  the  i n f e r*  Tice  contexts  with  which  we  are  dealing. 
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Finally,  the  program  was  so  complex  that  it  made  no  attempt  to  mimic 
the  time  required  for  solution  or  to  estimate  solution  times. 

Often  the  program  took  loncjei  to  solve*  the  problem  than  real  sub- 
jects did.  Human  problem  solving  behavior  is  so  idiosyncratic 
that  it  seems  unlikely  that  this  approach  would  provide  a produc- 
tive strategy  for  predictive  modelling  of  human  performance. 

2.4.3  Summary 

We  he.Yo  examined  as  carefully  and  realistically  as  possible 
the  prospects  for  developing  models  of  human  performance  based  on 
the  human  information  processing  modelling  literature . We  conclude; 

(1)  that  bits  and  pieces  of  relevant  models  exist,  (2)  that  there 
is  a rich  experimental  literature  that  can  provide  ideas  and  con- 
cepts tha t may  be  useful  for  building  human  operator  models  in 
specif ic  task  contexts,  but  (3)  that  no  generally  us. -ful  integrative 
models  exist  and  that  it  is  probably  p.  einufu  ‘ • to  try  to  develop  one. 

Advocating  the  use  of  the  information  processing  data  and 
modelling  literature  as  a base  presupposes  that  models  will  be 
developed  beginning  with  the  most  elemental  components  of 
performance  and  building  from  there.  If  this  approach  is  to  be 
taken,  the  most  promising  strategy  is  one  similar  to  that  being 
pursued  by  the  developers  of  1I0S.  We  should  pick  a particular 
task  or  collection  of  tasks,  analyze  (on  the  basis  of  the  best 
expertise  avn  ilr.Mo)  the  in  format  ion  processing  components  ol 
each  las’.,  and  assemble  a me 'del  fo*  each  t ask  in  such  a way  that 
th  • functional  components  or  modules  of  the  model  have  general il\ 
even  though  they  have  been  uniquely  combined  to  represent  only  one 
task.  These  task  models  should  then  be  validated  against  real 
human-operator  data  before  proceeding  to  the  next  level  of  model 
aggregation  across  several  tan1  . rich  an  approach  w.i  1 1 roquiregj  it 
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time  and  effort,  espec  i a 1 1 y early  in  its  development  when  the  most 
basic  components  are  being  formulated.  This  approach  should  pro- 
vide valid  and  general  representations  foi  particular  task  aggregates, 
such  as  the  RPV  control  problem,  but  would  probably  still  require  sub- 
stantial new. effort  to  apply  to  a new  problem  context. 

The  models  derived  by  human  information-processing  specialists 
appear  to  have  several  limitations  in  their  potential 
applicability  to  man-machine  system  modelling  problems. 

First,  they  tend  to  be  compartmentalized  by  the  very  fact 
that  most  of  them  deal  with  particular  stages  of  information 
processing  rather  than  being  integrative  of  human  performance 
more  generally.  While  this  may  make  them  useful  for  introduction 
as  component  models  in  a task-oriented  simulation,  one  faces  the 
same  problems  of  independence  and  interaction  at  this  component 
level  that  network  formulations  face  at  the  task  level  (see  Section 
3.3  , below) . Sternberg  (1969)  has  offered  the  additive  factors 

methodology  as  an  experimental  means  of  isolating  those  information- 
processing  stages  that  operate  independently,  but  in  fact,  the 
results  ar^  very  dependent  upon  the  experimental  task  or 
paradigm  in  which  they  are  measured,  and  one  has  no  assurance  that 
the  results  will  generalize  without  specific  validation  ia  each  new 
context . 

Second  , human  information  processing  model  s , for  the  most  part  , deal 
wi  1 the  average  performance  of  well -mot  ivatef:  highly  practices! 
individuals  under  relatively  ideal  conditions.  There  are  many 
hypotheses  but  few  data  and  virtually  no  models  in  the  inform  it  ion 
processing  literature1  on  how  human  performance  capacities  change 
under  str  -ss,  unties  reduced  mot  ivation,  before  practice  has 


1 

I 


.or t No.  344 b 


A i r T ra f £ i c Con t ro ] 


Bolt  Boranek  and  Kuw.na:i  Inc. 


to 


Our  review  has  disclosed  suri  risinqly  few  models  relevant 
him  an  performance  in  air  traffic  control  systems , though  the; 
literature  on  design  of  such  systems  and  on  estivates  of  the 
capacity  of  current  and  projected  systems  is  very  large.  V.'c  have 
included  in  the  Appendix  summaries  of  two  models  - one  concerned 
with  perception  of  conflict  and  one  with  controller  workload  - 
that  are  relevant  to  the  concerns  of  this  report.  (See  Abstract 
Nos.  32  and  33,  respectively.) 

The  model  of  conflict  perception  by  Dun] ay,  ot  al  (1973) 
deserves  particular  attention  because  of  the  author's  attempts 
to  address  the  discrepancy  between  prescribed  behavior,  as  set 
forth  in  formal  FAA  regulations,  and  actual  behavior,  as  determined 
from  study  of  controller  responses.  An  effort  is  made  to 
acconmodate  the  few  empirical  data  available  on  controllers' 
perceptions  of  aircraft  separation,  using  current  display 
hardware . The  model  indicates  that  collision  avoidance 
commands  can  be  expected  to  be  issued  prematurely  (as  compared 
with  the  dictates  of  the  regulations)  and,  hence,  lead  to  greater 
controller  workload  than  is  customarily  assumed  in  normative 
ATC  models. 


I 
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2.6  Models;  of  Tndusti  ini  Inspection 

Perhaps  because  of  their  ubiqui  ty  and  importance  in  industry, 
such  tasks  as  allocation  ol  resources,  network  capacity  planning, 
quality  control,  etc.,  have  been  of  increasin';  interest  to  operations 
researchers,  applied  mathematicians , and  human  factors  experts.  The 
result  of  this  interest  is  a growing  number  of  carefully  defined 
and  frequently  elegant  models. 

A large  percentage  of  the  models  we  have  reviewed  in  these 
areas  are  prescriptive  in  nature  - that  is,  they  are  concerned 
with  explication  of  optimal  performance  under  given  sets  of 
constraints  - rather  than  being  descriptive  of  actual  performance. 

This  characteristic  has,  in  our  judgment,  at  least  two  important 
ramif .ications  vis  a vis  application  to  modelling  and  simulation  of 
human  performance.  One  is  that  the  models  can  probably  be  effectively 
employed  only  after  the  design  of  a given  system  is  relatively 
complete  and  the  goals  of  and  constraints  or,  its  operation  can  be 
described  in  detailed  form. 

Secondly,  and  more  importantly,  since  many  of  those  models 
treat  the  human  on  a "black  box"  basis  and  do  not  attempt  to 
accommodate  empirical  data  relating  actual  performance  strengths 
and  limitations,  they  have  limited  value  in  identifying  the  most 
effective  uses  of  humans  in  complex  systems.  Although  one  might, 
as  t.he  result  of  exploiting  a part  icular  nr  -1,  gain  some 
understanding  of  the  best  that  can  be  hoped  fo-  from  a given 
system,  he  is  likely  to  be  left  uninformed  as  to  how  to  realize 
that  level  in  terms  of  real  combinations  or  • e:  and  machines. 
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Notable*  exceptions  to  these  generalizations  occur  in  the 
area  of  inspector  performance  where  , as  McCornack  (1961)  noted, 
in  his  review  of  the  literature,  that  the  occurrence  of  systematic 
rather  than  random  errors  is  cf  importance  to  the  serious  study 
of  inspector  accuracy.  Efforts  in  this  area  have  concentrated  on 
classifying  types  of  inspection  activity  - tor  example,  examining 
each  member  of  a set  of  similar  items  in  an  effort  to  .identify  one 
or  more  that  do  not  conform  vs.  examining  each  member  exhaustively 
for  defects  (see  Fox  and  Haselgrave , 1 969 )- on  empirical  determina- 
tion of  what  parameters  of  inspection  tasks  are  critical  in 
influencing  accuracy,  and  on  developing  and  applying  modelling 
approaches  that  explicitly  employ  cost/payoff  concepts. 

A theoretical  framework  that  has  proved  most  useful  in  the 
modelling  of  inspection  performance  is  that  of  signal  detection 
theory  (see  Abstract  No.  23).  This  framework,  first  applied  to 
industrial  inspection  by  Wallack  & Adams  (1969),  was  successfully 
used  by  Sheehan  and  Drury  (1971);  see  also  Drury,  1973)  to  study 
the  commission  of  "fcilse  alarm"  (classification  of  acceptable 
parts  as  rejcctable)  and  "missed  alarm"  (classification  of  re- 
joctable  parts  as  acceptable)  errors  among  a sot  of  quality 
control  inspectors  in  an  experiment  aimed  at  assessing  the  effect 
of  illumination,  visual  acuity,  and  learning  on  performance. 

Other  existing  models  have  been  adapted  to  predict  the 
results  of  inspector  performance.  Cochran,  Purswell,  and  Hoag 
(1973)  have  developed  four  models  based  on  Response  Surface 
Methodology  (see  Clark  and  Will  igos,  1973,  andwilliges  and  Mills, 
197!)  for  the  purpose  of  predict  ir..  the  relat  ! onsh  i ps  among  rale 
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of  change  of  visual  angle,  angular  velocity,  viewing  time, 
illumination,  and  contrast  in  dynamic  visual  inspection  tasks. 
Sheridan's  (19  6 9)  model  (see  Abstract  No.  34),  based  on  Bayesian 
concepts,  is  another  example  of  the  adaptation  of  frameworks 
developed  and  used  successfully  in  other  contexts. 

In  addition  to  modelling  techniques  brought  over  from 
other  areas  of  interest,  models  associated  uniquely  with  the 
inspection  and  process  control  environments  have  begun  to  appear 
in  the  literature.  One,  formulated  by  Drury  (see  Abstract  No.  35), 
accounts  for  an  earlier  finding  by  this  author  (Drury,  1973) 
that  increases  in  inspection  time  are  accompanied  not  only  by 
an  expected  decrease  in  the  acceptance  of  faulty  items  but 
also  by  an  increase  in  the  rejection  of  good  items.  A final 
example  in  this  category  is  the  model  by  Thomas  (1973;  see 
Abstract  No.  36),  which  incorporates  the  linear  relationship 
discovered  by  Hick  between  reaction  time  and  number  of  decision 
alternatives,  in  a model  designed  to  predict  performance  in  a 
parts  sorting  task. 
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2.7  Miscel  1 anoo'.i s .‘-lode* L s 

During  our  review,  wo  encountered  a number  of  models  that, 
ei idler  because  of  their  specific  formulation  or  because  of  t ho 
situations  to  which  they  apply,  were  of  only  tangential  interest 
to  the  context  of  this  report.  Some  of  these  models  are,  however, 
well-developed  and  serve  as  good  examples  of  modelling  technology 
that  might  be  tailored  to  the  needs  of  the  current,  project.  We 
have  included  four  such  examples  at  the  end  of  Appendix  A. 


The  models  discussed  by  Greening  (1976;  Abstract  No.  37)  have 
what  is  perhaps  the  greatest  potential  relevance  of  the  set 
contained  in  the  "Miscellaneous"  category.  In  particular,  they’ 
may  be  useful  in  the  model  ling  of  specific  performance  parameters 
associated  vith  visual  detection  and  recognition  of  targets  during 
routine  reconnaissa  a cj  dur u j the  tpons  delivery  phase  of  a 
i..  ; i e : . l he  x'ei.'.-.iin i ng  m 'dels  are  eon  •■•rn-d  with  human  performance 
in  systems  characterized  by  slow  dynamic  changes  (Rouse,  197  3, 
Abstract  No.  38),  with  decision  making  in  the  context  of  altern- 
atives that  are  dichotomous  with  respect  to  preference  value 
(Pol lay,  1970,  Abstract  No.  39),  and  with  workload  assessment 
under  conditions  in  which  attention  must  be  shared  between  con- 
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2.8  Gone ' a l Di scansion  and  u:.  >.  r y 

This  review  has  examined  a variety  of  models  derived  from 
a control -and  dec i si  on- theoret ic  framework,  the  modelling 
literature  in  .information  processing,  and  the  collection  of 
models  and  data  bank  formulations  originally  derived  from  the 
reliability  and  network-simulation  literature. 

It  should  be  clear  from  the  preceding  sections  that  manual 
control  of  vehicles  has  been  the  focus  of  a sustained  national 
research  program  directed  toward  the  development  of  human 
performance  models  of  practical  use  for  engineering  design,  and 
that  efforts  in  this  area  have  produced  some  well-defined  and 
successful  results.  There  is  also  a developmental  thread  in  the 
area  of  network  models  and  reliability  models,  but  these  models 
ave  not  been  as  broad-based  or  as  successful  thus  far.  The 
work  on  information  processing  models  has  been  sustained,  but 
its  orientation  has  been  toward  advancing  the  theory  of  human 
performance  rather  than  toward  practical  use,  and  this  work  has 
been  fragmented  and  compartmentalized  according  to  individual 
researcher's  personal  interests. 

The  most  successful  modelling  efforts  seem  to  have  grown 
out  of  situations  where  formal  models  of  the  task  environment 
are  well  developed,  such  as  in  feedback  control  tasks,  detection 
tasks,  and  well-defined  probubi.l  i s l ic  decision-making  tasks. 
Further,  in  these  areas,  the  most  successful  of  these  models 
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goals  and  success  criteria  used  by  human  cp  -raters  in  formal 
quantitative  terms.  One  difficulty  for  r,.  ailing  behavi or  in 
no  re  coraplo:-:  procedural  tasks  arises  from  zhe  inherently  multi- 
dimensional , multi-level,  time-varying  array  of  criteria  and 
strategies  that  an  operator  applies  in  accomplishing  these 
tasks . 


It  is  interesting  to  note  tlutu  the  optimal  control  model 
and  those  information-processing  models  derived  from  a decision- 
theoretic  framework  are  mutually  compatible ; this  suggests  the 
possibility  of  integrating  and  generalizing  them  to  provide  a 
single  modelling  framework  that  could  be  applied  to  vehicle 
control , supervisory  monitoring,  surveillance,  signal  identifi- 
cation, and  decision  making,  all  of  which  are  tasks  of  major 
interest  in  military  system  design  and  evaluation. 

As  we  view  the  modelling  literature  as  a whole,  the  most 
productive  work  is  in  the  area  of  signal  defection,  recognition, 
and  identification,  and  in  perceptual -motor  control,  where 
relatively  tight  feedback  loops  can  be  specified  and  human 
performance  is  relatively  constrained.  In  to.  area  of  intellectual 
performance,  modelling  efforts  have  not  produc'd  pracl ically 
useful  results,  either  in  areas  where  an  c xpl  icit.  algorithm  might 
be  specified  or  with  respect  to  general  pro;.!  em -solving  per- 
formance, where  a wide  range  oi  p<  • on  ai  • sti  itegies  a r<  • 
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A second  area  of  difficulty  concerns  what  Siecjel  and  Wolf 
refer  to  as  modulator  functions  (which  change  the  over-all 
efficiency  of  performance)  such  as  fatigue,  motivation,  practice 
stress,  or  individual  differences.  While  these  can  be  dealt  wit 
superficially  as  simple  modulators  of  performance  efficiency, 
such  a representation  is  largely  atheoretic  and  fails  to  take 
account  of  the  differential  effects  of  these  factors  on 
particular  aspects  of  performance  and  the  interactions  among 
processing  activities  that  these  differential  effects  imply. 

One  hopeful  sign  is  that,  for  many  purposes,  models  of 
performance  may  be  robust  with  respect  to  such  differences. 
Frequently  the  system  and  the  environment  impose  constraints 
on  human  performance  that  restrict  performance  variability 
directly  or  limit  the  range  of  strategies  or  methods  an  indi- 
vidual may  employ  if  mission  goals  are  to  be  met.  Frequently, 
models  can  be  set  up  to  take  advantage  of  these  constraints 
in  predicting  operator  performance,  and  one  finds  empirically 
that  operators  adapt  their  over-all  performance  in  accordance 
with  the  models.  Where  this  kind  of  mode]  is  feasible,  it 
may  eliminate  the  need  to  deal  directly  with  many  sources  of 
performance  variation. 

With  the  possible  except  Lon  o;  w • hu\  e it  •ntified  no 
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The  models  that  v.v  have  rev Lewed  Lst  at  many  different 
levels  of  specificity.  A recurring  flume  has  been  the  idea 
that  systems  models  need  to  be  built  at  a level  of  specificity 
appropriate  to  the  goals  of  the  modelling  effort,  and  these 
need  to  be  laid  out  carefully  in  advance.  We  believe  that  a 
network  modelling  language,  such  as  SAINT , is  adaptable  to 
any  desired  level  of  specificity,  ranging  from  the  elemental 
implementation  of  an  information-processing  activity  to  the 
more  global  representation  of  task  modules,  where  modules  ray 
be  segmented  at  arbitrary  levels.  In  Section  2.4,  wc  observed 
that  the  careful  segmentation  or  modularization  of  tasks  and/or 
task  components  is  of  overriding  importance,  is  made  even  more 
salient  when  we  look  over  the  diversity  of  modelling  concepts 
that  have  been  developed.  The  nature  of  the  taxonomy  cannot 
be  considered  without  regard  to  the  structure  of  the  model: 
themselves.  The  intrinsic  feedback  structure  of  control 
theory  models  requires  segmentation  of  a different  love!  than 
that  appropriate  for  building  tasks  out  of  information- 
processing components.  I l is  not  clear  wh«  . her  the  same 
hierarchy  of  structures  that,  is  appropr  ; • ‘ ■ to  i n lot  mat  ion- 
processing models  can  also  be  app  I it  ' •^"cvo  1 of  task- 

oriented  models  like  that  ol  Si  eg-  1 ar  ; W or  t lie  inspee  i on 
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In  summary,  we  bel  ieve  that  inteijr.il  ivo  motels  of  human 
performance  compatible  with  the  roqui remenf s for  representing 
command  and  control  performance  do  not  exist  at  the  present 
time.  What  is  available  is  a collection  of  bits  and  pieces 
taken  front  a variety  of  frameworks  that,  might  be  drawn 
together  to  build  an  eclectic  model  for  a particular  task 
situation  of  interest.  The  assembly  of  the  pieces  will  require 
substantial  effort  in  and  of  itself  and  is  likely  to  require 
many  assumptions  about  particular  aspects  of  performance.  If 
one  is  to  have  confidence  in  the  product  so  generated,  several 
iterative  validation  steps  at  different  levels  of  abstraction 
will  be  required. 

In  the  course  of  our  review  we  have  identified  several 
issues  relating  to  the  overall  perspective  and  implementation 
of  systems  models,  and  it  is  t:o  these  issues  that  we  turn 
next. 
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ISSUES  TO  BE  RESOLVED  II.'  APPLY 
MODELS 


.VS  HUMAN  PERFORMANCE 


3.1  Model  Validation 

The  problem  of  validating  a propos  d nnn-machine  model  is  a 
difficult  one  and,  indeed,  v;hnt  is  meant  by  model  validation  or 
verification  is  frequently  debated.  In  one  sense,  a model  is 
valid  if  it  can  be  used  to  arrive  at  reasonable  decisions;  in  such 
a case,  accuracy  may  not  be  very  important.  However,  with 
mathematical  or  simulation  models,  one  usually  looks  for  quanti- 
tative validation  or  tests  of  model  accuracy.  Even  in  this  case, 
there  arc  sometimes  two  views  as  to  what  is  meant  by  verification. 
One  view  suggests  that  a model  is  verified  when  the  model 
structure  and  parameters  can  bo  adjusted  to  provide  an  "adequate" 
match  to  experimental  data.  The  second  view  holds  that  a model 
is  validated  only  when  the  results  of  a new  experiment  are 
predicted  by  the  model  with  sufficient  accuracy.  Here,  we  consider 
the  first  process  to  be  one  of  model  identification, and  validation 
is  taken  to  mean  accurate  prediction.* 


For  either  view  of  model  validation,  it  is  necessary  to  compare 
experimental  results  w.i  th  model  predictions  and  to  apply  both 
engineering  and  formal  statistical  tests  to  determine  whether  or 
not  the  model  should  be  considered  "v.  ’id."  To  accomplish  this, 
one  must  make  judgments  concerning  (')  the  defin.it  ion  of  the  "data, 
(2)  the  appropriate?  f i gun  s-of-mer  i t foi  the  engineering  and 
statistical  tests , (3 ) the  si  rcific  statist ical  test  to  use , and 
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(4)  the  degree  of  discrepancy  between  experimental  and  model 
results  that  is  considered  acceptable. 

For  purposes  of  model  validation,  it  is  entirely’  proper  to 
consider  one  of  the  experimental  test  conditions  as  "baseline" 
data  for  the  purposes  of  adjusting  human  operator  parameters  of 
the  model.  In  this  way/ one  eliminates  the  possibility  of  rejecting 
the  model  because  of  differences  between  the  subject  pool 
available  for  the  validation  experiment  and  the  subject  population 
used  in  other  studies.  Instead,  one  can  concentrate  solely  on 
testing  the  ability  of  the  model  to  predict  performance  changes 
across  the  various  simulation  configurations  of  interest.* 

One  approach  that  is  commonly  used  for  statistical  testing 
follows  the  "null  hypothesis."  Specifically',  one  adopts  an 
initial  hypothesis  that  discrepancies  between  experimental  results 
and  model  predictions  are  the  result  of  "experimental  error"  (that  is, 
the  result  of  inherent  variability  in  the  data  and  not  of  inaccuracies 
of  the  model).  A formal  statistical  test  is  then  performed  to 
determine  the  probability  that  such  discrepancies  are  due  to 
experimental  error.  If  this  probability  is  sufficiently  low 
(i.c.,  below  some  "confidence  limit"),  one  concludes  that  model 
inaccuracies  are  statistically  significant  and  the  model  is 
"rejected."  Otherwise,  the  model  is  considered  valid  (more 
precisely,  we  fail  to  call  it  invalid). 
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One  of  the-  i nevi  table  problems  vi  th  this  type  of  testin';  is 
that  the  power  of  the  test  depends  or.  the  ana lity  of  the  experi- 


mental data.  For  example,  if  the  subject  pool  is  not  sufficiently 
la rye  and/or  if  subject-to-subjoct  variability  is  very  qreat,  it 
may  be  difficult  to  reject  the  model  unless  it  .is  totally 
unreasonable.  That  is,  we  do  not  have  a sensitive  test  to  apply 
to  competing  models. 

Therefore,  the  experimenter  must  conduct  a very  carefully 
controlled  experiment  so  that  the  statistical  test  is  reasonably 
discriminatory.  Frequently,  this  is  r.ot  a major  problem  for 
well-trained,  well-motivated  subjects  performing  well-defined, 
demanding  tasks.  For  example,  in  laboratory  experiments  in  manual 
control  it  is  often  the  case  that  four  test  subjects  provide 
sufficiently  consistent  data  so  that  performance  differences  on 
the  order  of  15i  can  be  considered  tc  be  significantly  different 
at  the  0.05  level  of  confidence.  A somewhat  larger  subject  pool 
may  be  desired  for  more  complex  C tasks,  since  one  expects  subject 
variability  to  increase  as  the  number  of  performance  dimensions 
increases . 

The  choice  of  a confidence  limit  for  rejection  of  the  model 
depends  on  the  tradeoff  one  inak.es  between  the  probability  of 
rejecting  a model  when  differences  are  actually  due  to  experiment  a 1 
error,  and  the  probability  of  accej  the  model  when  it  is  truly 

inaccurate.  The  lower  the  confidence  limit,  Mie  greatei  must  ! 
the  discrepancies  to  cause  rejection.  Typically,  confidence 
limits  are  not  in  the  range  of  ().  0 i *.  0.05. 
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it  is  also  possible  that  in  some;  unusi.  cases  the  statistical 
test  may  prove  too  strong.  For  example,  if  the  subjects  arc 
extremely  consistent  in  their  performance,  and  if  the  subject  pool 
is  large,  performance  differences  as  low  as  10  could  be  declared 
statistically  significant,  leadiny  to  rejection  of  the  model. 
Accordingly,  it  is  important  that  a valid  engineering  test  be 
applied  along  with  the  formal  statistical  test. 


Since  there  are  a number  of  statistical  tests  one  might  use 
to  validate  a model,  one  cannot  define  a "best”  procedure.  One 
reasonable  approach  to  statistical  model  validation  is  to  apply 
t-tests  to  all  relevant  performance  measures.  In  terms  of  a 
validation  experiment 


x-;i 


t = 


N ( N - 1 ) 
s s 


N 

>:s 

i=l 


[>:  j-p]  2> 


1/2 


where  N is  the  number  of  subjects,  x.  is  the  ensemble  average 

S 1 

of  the  particular  performance  measure  foT  the  i-th  subject,  x is 
the  average  performance  across  all  subjects,  and  u is  the  model's 
prediction  of  the  correspondi  ng  ensemble  statistic,  x ( t ) . 

Note  that  a separate  test  is  performed  for  each  ensemble 
variable  of  interest.  Rejection  or  acceptar.ce  of  the  model  along 
this  performance  dimension  might  then  be  determined  bv  the  variable 
that  is  least  well  predicted.  Vital  is,  the  model  is  rejected  if 
the  t-tests  reveal  at  least  one  measure  that  cannot  be  alti ibutod 
to  experimental  error. 
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is  often  used  as  a possible 


where  E is  the  measure  of  prediction  error,  N is  the  number  of 
variables  of  interest,  and  x is  the  data  ensemble  statistic  for  which 
y is  the  corresponding  model  prediction.  The  summation  is  taken 
over  all  matched  variables.  Thus,  E can  be  interpreted  as  the 
average  squared  difference  in  terms  of  percentage  mismatch. 

Alternatively,  if  sufficient  data  are  available  from 
different  subjects  to  compute  the  inter-subject  variability, 
then  an  appropriate  engineering  f igure-of-merit  for  a given 
performance  variable  is  a standard  score,  computed  as 

Z-  = 
x 

cr- 

x 

where  x and  y are  defined  as  before  and  c-  is  the  standard 
deviation  (across  subjects)  of  the  performance  measure  x.  If 
Z £ 1,  then  we  can  conclude  that  the  model  is  about  as  good  a 
predictor  of  subject  performance  as  is  another  subject.  This  is 
probably  all  one  can  hope  for.  Of  course,  one  has  the  option  of 
averaging  the  Z's  for  different  performance  measures  or  of  treating 
them  independently. 

As  a last  point,  we  consider  the  situation  where  the  sair :•-* 
performance  variables  are  measured  for  a number  of  different  ex- 
perimental conditions.  One  can  consider  the  deviations  of  these 
measurements  from  model  predictions  in  a manner  similar  to  the 
alAive  treatments.  Alternatively,  one  could  use  the  correlation 
statistic  as  a quantitative'  means  of  examining  the  correspondence 
between  model  predictions  and  emp _ rically-der i vid  results. 


For 
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each  performance  measure,  one  would  compute  the  correlation  coeffi- 
cient between  observed  performance  and  predicted  performance 
across  a set  of  experimental  conditions  that  give  rise  to  a suitable 
range  of  variation  in  the  measure  under  study.  To  provide  the 
strongest  test  of  the  model,  one  would  want  to  either  fix  the 
parameters  of  the  human  operator  portions  of  the  model  and  vary 
only  those  parameters  that  are  associated  with  the  experimental 
conditions  under  study  or  at  least  to  sel ect  human  operator  param- 
eters according  to  rules  defined  in  advance.  The  more  human 
operator  parameters  that  are  adjusted  specifically  to  match  the 
data,  the  easier  it  should  be  to  obtain  high  correlations. 


For  models  of  complexity  sufficient  to  represent  full-scale 
man-machine  system  performance,  the  problems  of  validation  go 
well  beyond  the  selection  of  the  proper  goodness-of-f it  statistics. 
If  the  model  is  to  be  useful  as  a design  tool,  it  must  be  validated 
prior  to,  or  at  least  concurrently  with,  the  development  of  a 
full-scale  simulation.  Under  these  conditions,  system  performance 
data  against  which  to  validate  the  model  are  not  likely  to  be 

( 

available,  and  one  must  seek  less  rigorous  means  of  examining  the 
validity  of  the  model.  In  the  course  of  building  the  model, the 
investigator  will  inevitably  generate  qualitative  hypotheses  about 
how  behavior  will  change  with  changes  in  certain  critical  system 
parameters.  At  a minimum,  one  should  be  able  to  predict  the 
direction  of  changes  in  sysfom  performance  measures  if  not  their 
magnitude.  Thus  one  simple  step  toward  validation  would  involve 
testing  the  mode. I against  such  system  parameter  changes  to  ensure 
that  it.  responds  in  accordance  with  those  prior  expectations.  It 
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parameter  changes  will  tell  us  only  whether  the  model  is  success- 
ful (an  unlikely  outcome  on  the  first  try)  ,*  but  not  what  to  do 
to  improve  it.  The  empirical  validation  must  either  proceed  in 
steps  from  specific  to  general,  as  suggested  in  Section  2.4,  or 
else  provide  data  concerning  performance  at  all  levels  in  the 
hierarchical  structure  of  the  model  and  in  terms  of  the 
identifiable  model  parameter^  at  each  level. 

Finally,  there  is  a logical  paradox  that  arises  when  attempting  to 
validate  any  predictive  model,  the  purpose  of  which  is  to  describe 
performance  in  some  new  situation;  if  we  experimentally  examine 
conditions  sufficiently  close  to  that  n ew  situation,  we  have 
eliminated  the  need  for  prediction,  since  we  would  then  have  available 
experimental  data  of  direct  usefulness.  Perhaps  the  goal  of 
validation  should  be  to  span  the  range  of  conditions  of  application 
so  that  prediction  becomes  a task  of  interpolation  in  a multi- 
dimensional parameter  space  whose  boundaries  have  been  explored, 
rather  than  an  extrapolation  to  a wholly  new  domain  of  activities. 

In  summary,  the  problem  of  validating  large  sea le  systems  models 
is  not  one  for  which  there  are  always  textbook  answers.  It  is 
an  issue  that  must  be  addressed  far  each  new  model  that  is 
created,  and  further  effort  is  required  ro  develop  guidelines  for 
how  to  procee.  in  particular  cases. 

I 
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3.2  Selecting  the  Level  at  Which  to  Formulate  Man-Machine 

Per  forma nee  Model s 

There  are  several  issues  that  interact  in  the  attempt  to 
select  the  appropriate  modelling  approach  to  solve  a given  system 
evaluation  problem. 

The  overall  goal  should  be  to  develop  a model  that  has 
sufficient  generality  to  be  able  to  predict  system  performance 
over  a range  of  parameter  settings  for  classes  of  parameters  or 
attributes  for  which  the  designers  have  some  leeway  in  design 
and  for  which  they  are  uncertain  regarding  suitable  values.  It 
is  very  difficult,  if  not  impossible,  to  develop  a model  that 
will  serve  as  a useful  design  tool  for  the  study  of  circumstances 
that  were  not  anticipated  at  the  time  of  its  conception  without 
some  modification  or  adjustment  along  the  way. 


Thus  a modelling  effort  should  begin  with  a clear  definition 
of  the  purposes  for  which  it  will  be  used  and  the  conditions 
under  which  validity  is  desired.  A critical  part  of  this  defin- 
ition is  the  level  of  detail  at  which  prediction  is  desired.  If 
the  primary  objective  of  a model  of  an  RPV  Control  System  is  to 
predict:  the  < : f fe  z t s on  w »ap  ms  • ' ?1  i v ■ r pei  foifman  • ■ of  chat  jes  in 
communion t ' 'v:.'  ba  iov.»  id  fit  between  the  control  station  and  the 
vehicle  there  is  no  need  to  represent  human  performance  at  the 
level  of  detail  that  would  be  sensitive  to  changes  in  workplace 
layout.  A model  that  is  to  evaluate  op<  rator  v rrkload,  however, 
may  require  a representation  of  the  workload  associated  with 
the  operation  of  a .light  pen  that  can  be  invoked  each  t imo  a 
1 ight  pen  is  ued , but  i I need  not  rep re sen!  the  ] srceptual-motor 
details  ol  light  pen  operat  ion.  On  the  other  hind,  if  a denim 


i 


— — 


change  will  Impact  on  the  w ly  a igl ts  1 as  opposed 

to  the  number  of  tii  ss  it  is  used,  then  a n r<  del  iil< 
representation  may  be  i juired . We  kno w < > - s i - g ieral 
procedures  that  wil  1 replace  thoughtful  analysis  for  arriving 
at  a judgment  of  the  suitable  level  at  which  to  model. 

The  question  of  level  of  detail  interacts  with  another 
issue  which  remains  to  be  resolved:  when  modelling 

a complex  man-raacni  n«.-  system,  does  one  proceed  to  develop  a 
blo~k  dia m a;  at  the  most  general  level  and  then  break  it 
do i into  : •••«•  d ’'ailed  ! cvels  as  necessary  or  does  one  start 
at  * ' :n  . fun  i ■ ! il  level  of  behavior  and  systematically 

int  • ; runts  to  build  the  mode  12  These  two 

up  - : ir.  <;  referred  to  as  the  top-down  vs. 

!>  ■■■  1 or  the  analytic  vs.  synthetic.  Although 

cl  di  d . cannot  Lie  made,  the  optimal  control  model 

h r .n  o:  th  • ivor  of  the  top-down  approach  whereas  the 

ii  . s : mil  at  .mis  are  representative  of  a bottom-up  approach. 


It  might  be  argued  that:  a detailed  analysis  of  the 
eler ental  components  of  performance,  carefully  represented  and 
validated, could  then  bo  built  up  systematically  into  a 
more  integrated  model,  as  was  argued  in  the  discussion  of 
information  processing  models.  if  all  behavior  were  purely 
additive  in  time  ind  n iltiplicat Lve  in  accuracy,  this  Idea  might 

ractic  i.  \ fortunately  this  is  not  the  case.  Behavioral  scientists 
hr  o a 1 ways  boon  curv'd  by  statistical  interactions  among 
variables  of  in t er est,  and  such  interactions  usually  must  be 
mod-  lied  explicitly  at.  each  new  level  of  integration  that 
is  mder taken.  Because  oi  human  strategy  variations  and  other 
Vi  : .tries  of  human  performance,  it  simply  is  not  possible  to 
bl  1 v as.  iic  aMitiv-'  and  m 1 1 ' ipl  ieat  i ve  comb’ rial  Lc»n  rule.: 


, i r. r i or  i t heoret  i nl  i 
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in  Soo  t.  ion  i.  1 . It  in.  this  di  i ficulty  with  s*  rategy  variations 
that  Limi  s th  ipj  it  :nt  general ity  of  a bottom-uj  model . As 
si  . one  ■ . situations , ne .-:  strategies  emi  rge  t h 1 1 

rt'iu  : t-o  .-a  just:  or  r • • ornulat  ion  of  the:  mode]  . 

Perhaps  th-  most  difficult  challenge  of  the  bottom-up 
approach  is  th<  nro mclio:;  of  a suitable  general  tax-  noiny  for 
deriving  the  components  of  human  performance  out  of  which  the 
tasks  are  to  be  reconstructed.  Unfortunately,  we  still  do  not 
have  suitable  procedures  for  developing  such  a taxonomy  on  any 
basis  other  than  in:ormed  intuition.  This  is  an  area  in  which 
much  work  is  needed,  bat  it  .is  not  clear  whether  a method  can  be 
derived  or  whether  the  difficulty  is  simply  reflective  of  our 
lack  of  theoretical  .understanding  of  the  components  of  human 

’ S"  )• 

performance . 


If  the  bottom-up  approach  is  thus  restricted  in  generality, 
what  of  the  top-down  alternative?  In  this  case  one  begins  with 
a statement  of  goals  or  objective  functions  which  the  system  is 
to  minimize,  a description  of  the  design  parameters,  and  the 
range  of  values  over  which  generality  is  sought.  The  designer 
moves  down  the  hierarchy  of  cicala  and  subgoals  only  to  the  level 
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Finally , because  top-down  model:  do  not  jet  t > certain  elemental 
lev  'Is  o f mi >dolling , they  are  frequently  unable  to  predict  th< 
effects  of  chan. 70s  that  arc  the  traditional  concern  of  human 
factors  specialists  (e.g.,  display  readability,  control -display 
compatabi litv , etc.) 

Top  down  models  reduce  the  tc  Mono  lie  problem,  but  in  its; 
place  a critical  need  arises  for  well-defined  dimensions  or 
parameters  of  system  performance  from  which  to  formulate 
performance  indices.  It  may  be  just  as  difficult  to  derive 
these  for  human  performance  characteristics  as  it  is  to 
derive  the  taxonomy. 

We  do  not  see  this  top-down  vs.  bottom-up  issue  as  one 
having  a general  solution,  but  rather  as  ono  for  which 
accumulated  experience  and  validating  data  would  help  to 
guide  the  model  builders  working  in  this  area. 


3.3  Approaches  to  Ha;  iling  r nteracti  ns  i 01  j 


Jt  was  observed  in  connection  with  netv.'or/.  appro  mhos  to 
predict  ion  that  there  are  fev»  pi  Luci.plos  to  cju : do  the  selection 
of  composite  reliability  and  time  distribution;;.  This  matter  is 
of  concern  where  tasks  or  behaviors  occurring  in  a procedure 
sequence  are  not  independent  and,  hence,  the  standard  product  and 
summation  rules  o:  reliability  theory  are  inappropriate. 

In  such  circumstances,  application  of  these  rules  may  result  in 
predictive  estimates  that  deviate  critically  from  those  derived 
I from  actual  observation,  as  demonstrated  by  Buckner  and  McGrath 

j (1961;  see  also  Swain,  1964). 

The  significance  of  the  problem  has  been  noted  by  a number 
of  investigators  (Lamb,  1 970;  Meisler,  1971;  Swain,  .1  967; 

Mills  and  Hatfield,  1974)  and  with  respect  to  the  product  rule, 
there  appears  to  bo  general  agreement  over  four  possible  solutions : 
1)  Find  means  for  identifying  and  qualifying  task  dependencies 

I and  then  reformulate  the  product  rule  to  inclxide  a 

dependency  term. 


2) 

Limit  u • j o of 

the 

rule  to  situations  whore  dependencies 

do  not  exist 

. 

3) 

Limit  use  of 

the 

rule  to  situations  in  which 

large  errors 

in  ; 

_>r edict  i ve  accuracy  are  tolerable. 

4) 

Identify  a 1- 

eve  1 

of  performance  modelling  at 

which  dependencies  do  no l exist  and  the  product  rule 
can  be  used. 

01  tl  >e , ; rhaps  onl>  ( l ) ind  (4 ) arc  solutions  in  the 
rr,.'  sons-  of  th  term,  since  to  limit  modelling  activity  to 
: • ' lorn:;  to  dependencies-  do  not  exist  or  to  those  where 
accuracy  not  :ri t ieal  bo  to  c<  1 t < i ; i sox  :relj  thi  si  1 

of  : a n/m  ■ h i n . • .••••.items,  one  can  In  >;  .•»  t'O  'duly.  As  a i < ui:  1 t , 
efforts  to  sol  --  the  d.  ; ■ ■ 1 1 < i -nc.'  ; : . • 1 > 1 • . usually  empire.  • ?.e  e i t he  r 
t h • •••all  e i 1 1 > t < c . 1 a > 1 1 i ' . . i • : ■ .-,,<•■  r .'  1 h . ■ o'  t a •,). 

in-,  r action,  f - r t lie  pur;  ••  ••  o'  as.ei  1 ; • ■ no  a • : ipr  into 

e I 1 ; ( • ■ *t  - :|l  I " It  S , 1 < 0 ; Mill  )*)d 
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Both  o i these  appr<  ches  ha\  th*  r strei  • • hi  nd  weakn<  . 

The  Mills  and  i.  i • Leld  proposal,  whir  . nv  ’ . • pro  . 

parametric  studies?  that  examines  in  areal  det  iil  the  s<  sntial 
dependencies  among  ongoing  series  of  task  e lemon  * . and  seeks 
to  find  distributions  that;  adequately  character  i ze  cornbinat  ions  of 
these  dependencies,  holds  the  promise  t yielding  results  that 
enable  very  precise  prediction  of  performance.  At  the  same  time, 
the  results  may  lack  generality  unless  the  program  of  research  is 
exhaustive  and  includes  the  study  of  t tsk  performance  in  its  full 
dimensionality  - that  Ls , with  the  accoj  >an intent  of  motiv.  ional  f ictors , 
stress,  learning,  etc.,  normally  encou; tered  under  actual  field 
conditions.  Swain's  approach,  on  the  other  hand,  appears  to 
promise  generality  but.  requires  .a  degree  of  understanding  of  the 
detailed  tasks  t.nd  procedures  to  be  pursued  in  a given  system  and 
a level  of  "artistry"  in  applying  past  experience v; ith  si  .. i 1 n r tasks 
and  procedures  that  may  preclude  successful  exploitation  of  the* 
modelling  technique  by  less-than-totally- .informed  analysts. 


Our  review  has  disclosed  little  that  would  suegest  that  , . t 
this  point  in  the  development  of  modelling  technology,  one  of 
these  approaches  is  to  be  preferred  over  the  other.  Th*.  choi  <: 
among  possible  combinatorial  rules  tnd  distributions  in  so 
inf  imatel  y tied  to  <juest  ions  concerning  the  le  • * 1 of  j t • a i ct  i v- 
a :cui  cy  desired , the  adequacy  ol  av;  ; L ab  1 e pr<  :ess/t 
i : no  i s , .inn  int  r i ns  i c 4 1 1 1 . i . - 1 ■ r Lcs  of  • • j • :u  1 at 
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3.4  Undordeteriiiir  >*- ion  o Mu  1 1 i-P  u ;toi  M i 1 a 

In  ci  typical  v:n  1 1 - f arrv  1 1 a tod  mathematical  model  of  a well- 
defined  ; voce"iS,  a goal  of  th  ; modeller  is  to  represent  behavior 
with  as  few  free  pm  .inters  as  possible.  Ideally,  there  should 
be  fewer  such  parameters  than  there  are  dependent  variables 
to  be  predicted.  This  principle  has  been  expressed  by  the 
epigram,  "If  we  model  the  muscle  system  with  69  parameters  i t 
is  possible  to  make  a muscle  spindle  sing,  ’God  Bless  America.'" 

When  wo  move  to  models  of  complex,  highly- interactive  man- 
machine  systems,  the  issue  is  much  harder  to  pin  down.  Model 
parameters  generally  fall  into  four  classes:  (1)  Parameters  that 

are  defined  by  the  initial  conditions  under  study,  such  as  hardware 
variables  for  which  specification  forms  a part  of  the  problem 
statement.  (2)  Parameters  that  form  an  integral  part  of  the  model, 
but.  that  may  be  assumed  to  be  invariant  over  the  range  of  condi- 
tions to  be  studied.  Their  values  may  be  estimated  by  validating 
experiments,  by  theory,  or  by  assumption,  but  t hey  arc'  not  free  to 
vary  fr<  . run  to  run.  (3 ) Parameters  t Ijat  may  vary  from  cond.it  i < >•■ 
to  condition,  but  for  which  theory  or  experiment  defines  the  rules 
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An  interesting  issue?  for  whi  r.  we  know  of  no  cjuiding  principles 
is  just  how  constrained  such  a mod  .oust  be  in  order  to  make  useful 
generalizations.  Is  there  a reasonable  ratio  of  unconstrained 
parameters  to  dependent  variables  rent  leads  to  useful  models? 

How  much  uncertainty  i n paran -_tex*  settings  can  be  tolerated  before 
the  predictions  lose  credibility?  It  seems  likely  that  some  of 
these  questions  depend  on  the  particular  model  or  application 
domain,  but  we  believe  some  genera,  statements  might  be  made  as 
we  accumulate  experience  with  alternative  model  forms. 

Simple  models  have  the  potential  for  exploiting  statistical 
procedures  for  identifying  parameter  values  to  maximize  the 
goodness  of  fit  to  a data  set;  however,  simulation  models  of  the 
scope  considered  here  can  become  a "fiddler's  paradise"  because  of 
the  complex  interactions  involved.  Is  it  possible  to  devise 
systematic  approaches  to  estimate  r arameters  that  do  not  have 
full  statistical  rigor,  or  ever,  the  rigor  of  efficient  hill-climbing 
algorithms,  but  yet  provide  some  bounds  on  the  time,  effort,  and 
confidence  in  the  values  obtained?  We  reuard  this  also  as  a 
researchable  issue. 
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In  this  report  wo  have  considered  a ] a;  ge  nur  r o'  human 
performance  models  and  modelling  approaches.  Further,  we  have 
considered  a family  of  issues  that  surround  attempts  to  employ 
these  models  in  the  predication  of  task  performance.  In  this 
section  we  summarize  our  recommendations  for  future  work  concerning 
modelling  of  multi-man  command  and  control  systems.  These 
recommendations  include  development  of  methodological  tools  and 
concepts  as  well  as  specific  extensions  of  existing  modelling 
technology. 

4.1  Develop  a Test-Bed  Facility  in  Which  to  Evaluate  and  Validate 

Models  on  a Comparative  Basis 

While  many  models  have  been  developed  from  a variety  of 
perspectives  focusing  on  a variety  of  issues,  v/e  know  of  only  a 
few  cases  in  which  specific  model  validation  studies  have  been 
undertaken,  and  we  know  of  no  cases  in  the  system  context  in  which 
the  same  operations  have  been  modelled  from  different  approaches 
in  order  to  compare  the  relative  effectiveness  of  the  approaches 
employed.  It  is  with  these  goals  in  mind  that  we  propose  develop- 
ment of  a generalized  test-bed  facility. 

At  this  point,  we  have  not  identified  in  detail  the  critical 
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operating  system  in  which  both  mod.  in  and  live  simulations  can 
be  rapidly  constructed  and  debugged. 

We  envision  four  modular  components  to  the  operating  system: 
(1)  a module  for  implement ing  the  hardware  components  of  the 
system  exclusive  of  the  specific  displays  and  controls  with  which 
human  operators  would  interact;  (2)  a module  that  provides  the 
software  linkages  to  the  specific  display  and  control  hardware  to 
be  used  in  manned  simulations;  (3)  a module  in  which  to  implement 
models  of  human  operator  performance  that  takes  as  its  input  the 
same  information  that:  would  be  input  to  the  display  hardware  and 
provides  outputs  identical  to  those  produced  from  the  operators’ 
controls  (this  module  would  thus  permit  the  same  system  hardware 
simulation  to  be  used  interchangeably  with  both  manned  simulation 
and  models  of  the  human  operators’  performance);  and  (4)  an 
automated  performance  measurement  module  that  would  make  it  easy 
to  program  the  data  collection  and  analysis  activities  associated 
with  either  mode  of  operation  so  that  the  same  measures  could  be 
collected  in  both  cases  when  comparisons  are  to  be  made.  In 
addition  to  the  direct  comparability  that  such  an  implementation 
would  afford,  a further  advantage  would  be?  the  ability  to  replace 
human  performances  of  particular  tasks  or  perhaps  of  one  operator 
of  a team  with  real  time  models  of  that  performance,  maintaining 
the  rest  of  the  simulation  inlnot. 


One  of  the  primary  uses  we  envision  for  the  propos.  d test- 
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For  those  modelling  approaches  t hat  involve  synthesiz  i nq 
higher- level  models  from  elemental  components,  we  would  urge  that 
a process  of  incremental  model-building  and  incremental  validation 
be  studied.  This  process  would  involve  validating  each  of  the 
model  subcomponents  as  they  are  developed.  We  believe  that  this 
approach  would  yield  several  benefits  in  terms  of  both  accuracy 
and  simplicity.  By  conducting  sensitivity  analyses  for  various 
subcomponent  parameters,  one  can  probably  reduce  the  number  of 
variables  that  must  be  carried  along  .into  the  higher-level  compon- 
ents. If  this  turns  out  to  be  the  case  for  a wide  enough  variety 
of  contexts,  the  "fiddler's  paradise"  problem  noted  in  the  previous 
section  may  be  substantially  reduced. 

A second  benefit  of  this  approach  is  that  it  provides  a 
vehicle  for  systematic  study  of  combinatorial  problems  associated 
with  the  aggregation  of  non-independent  sub-tasks.  As  suggested 
at  a number  of  points  in  this  report,  we  consider  these  problems 
to  be  among  the  most  critical  facing  successful  exploitation  of 
the  bottom-up  approach.  Further,  we  believe  that  if  these  problems 
cannot  be  solved  in  principle  for  a significant  number  of  system- 
relevant subtasks  and  their  interrelationships,  this  approach  may 
eventually  give  way  to  approaches  that  do  not  require  the  synthesis 
of  subtask  models. 
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review  of  the  human  performance  moduli.  Lru;  literature.  The 
incremental  modelling  approach , pursued  over  a set  of  well-defined 
case  studies,  should  enable  one  to  yain  a very  clear  understanding 
of  the  families  of  combinatorial  rules  and  distributions  that 
would  be  of  general  utility  in  botton-up  modelling  efforts. 

Further,  it  should  provide  one  with  some  feeling  for  the  loss  of 
accuracy  occasioned  by  the  use  of  simpler  combination  rules  and 
distributions  drawn  from  standard  reliability  theory. 

Finally,  the  incremental  approach  offers  a context  in  which 
to  develop  criteria  and  methods  for  fabrication  of  process/task 
taxonomies.  One  might  hope  to  learn  from  these  efforts  whether  it 
.is  possible  to  develop  a single  strategy  for  partitioning  tasks 
and  to  employ  successfully  a single  generalized  taxonomy,  or  whether 
it  is  necessary  to  evolve  set  of  strategies  and  taxonomic 
procedures . 

We  cannot  overemphasize  our-  fee-ling  that  the  generation  of  a 
testbed  facility  at  this  time  is  critical  to  the  continued  develop- 
ment of  human  performance  modelling.  We  believe  that  it  provides 
the  only  mechanism  by  which  important  issues  can  be  clarified  and 
needs  for  further  data,  theory  and  research  identified. 

4.2  Research  Recornn  tndai ions  Having  Applicability  Independ  ;nt  ol 

Me  del  1 in  i Approach  or  Topical  Area 


4.2.1  Co  oonent  Model  Aggregut  on 

The  use  of  r:  .dels  embedded  in  a Target  network  of  tasks,  whether 
they  be  mod  -1  s of  ‘asks  or  model  sof  component  information  processing 
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on  the  overall  accuracy  o:  r./stom  i ••vel  predi  etions  as  a function 
of  th  • number  of  a tons  of  aggrogati  . Sv,  , in,  (1969)  has  provided 
a starting  point  for  such  a study  i : r h • area  of  reliability 
aggregation,  but  v;e  believe  the  qv.  it  ion  could  be-  addressed  more 
generally  to  include  propagation  c.  other  kinds  of  modelling  error 
in  both  time  and  accuracy. 

The  modelling  errors  may  be  caused  by  biases  in  component 
predictions  or  by  incorrect  assumptions  concerning  the  form  or 
variability  of  the  assumed  component  performance  distributions. 
Some  kinds  of  distributions  may  be  more  robust  with  respect  to 
aggregation  than  others.  Mills  and  Hatfield  (1974)  provide  a 
starting  point  for  this  exploration. 

A further  aspect  of  error  propagation  relates  to  the 
structural  interrelations  among  the  component-  models  and  whether 
or  r.ot  the  interactions  between  components  are  considered 
explicitly,  implicitly  or  neglected  altogether.  A systematic 
study  of  these  influences  would  contribute  ge.nerically  to  future 
model  development. 

4.2.  ? V i lie  at  ion  Research. 

As  suggested  in  Section  1.1,  the  validation  of  models  of  the 
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4.2.3  Empirical  Eva lu a t i on  of  the  Proble  . of  . irameti  : 

Und e i ■ : t.  ' r m i n it  Lon 

Research  Is  needed  to  examine  the  impact  of  trade-offs  between 
the  number  of  parameters  that  must  be  estimated  from  data  in  live 
simulation  runs  and  the  number  of:  system  performance  measures  to  be 
predicted  from  the  model  of  human  performance  in  the  simulation. 

4.3  Research  Recommendations  Concerning  Topical  Areas  for  Model 

Develop  ament 

We  have  identified  two  areas  in  the  continued  evaluation  of 
man/machine  systems  where  we  believe  there  will  shortly  be  a 
crucial  need  for  modelling  techniques.  These  are  discussed  briefly 
below. 

4.3.1  Supervisory  Control  and  Monitor ing 

Many  of  the  activities  that  are  performed  by  men  in  command 
and  control  systems  may  be  thought  of  as  supervisory  monitoring 
and  controlling  activities.  While  applicable  integrative  models 
for  these  processes  are  not  currently  available,  many  of  the 
concepts  and  principles  have  been  introduced  either  .in  the  manual 
control  literature  or  in  the  information  processing  literature. 

A recent  NATO  Conference  (Sheridan,  1976)  focussed  attention 
on  this  topic,  and  we  believe  that  substantial  progress  can  be  made. 
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4.3.2  Team  Per formance 

Though  conceptualizations  of  group  performance  abound  in  the 
literatures  of  social  psychology  ar.d  management,  few  are  sufficiently 
quantitative  in  nature  to  be  applicable  to  descriptive  modelling  of 
team  performance  in  complex  man-machine  systems.  For  purposes  of 
simulation,  the  Siegel  & Wolf  approach  mentioned  briefly  in  this 
report  represents  perhaps  the  most  comprehensive  attack  to  date  on 
multidimensional  aspects  c crew  performance. 

We  believe  that  the  successful  development  of  team  performance 
modelling  will  be  aided  by  inputs  from  at  least  two  sources:  (1) 

a reasonably  comprehensive  empirical  data  biise,  and  (2)  a concep- 
tualization of  team  activity  that  clearly  distinguishes  those 
components  of  the  total  variance  in  performance  that  are  associated 
uniquely  with  the  interaction  of  team  members  from  those  that  are 
associated  with  members  acting  individually;  that,  is,  a conceptual- 
ization of  the  genera]  form 


N 


(L 

= o 

+ 0 

+ 

0 

n-1  n 

t 

c 

w i i e t? 

T 

the  total  vari 

mce  in 

perf 

ormneo 

n 

= the  variance  ir 

total 

team 

performance  due  to 

member  n 

t 

the  variance  ii 

total 

team 

t>e 

rformanoe  due  to 

interact  i or 

e 

- the  error  vari: 

nee 

among  momhc’ 

s 

t:  f f o r t 

c3 

to  meet  these  nee. 

:■  shoil  1 

a pr 

>C( 

i concurrently  a 

nd  with 

the  pi- 

'i a 

a tie  goa  1 o’  dot- 

rm  i nine 

ha 

t c 

i iuict  ct  i s i i cs  ci r 

t lu 

pe  i f or 

in  a o i i 1 - 1 son  tc 

a ! • can 

I V * 

i ' * 

■ • fully  be  i k!<  - 1 1 

d by 

comb  i n i n 

.1  t’no  outputs  of 

In  t i an; 

: od 

c 1 s 

< r i nd  i v i du:«i  1 b 

ha  vi  r 

Jh  nuclei  isti  \:  • • ;i  i . • •••  eci  lie  . »ou:  ‘V.  ...»  ,• 


J.44G 


Bolt  15  er 


an  Inc. 


!•’.  ••  >rt  Mo. 


Wo  believe  that  the  generation  of  group  performance  data  and 
lels  mi  usefully  begin  in  the  areas  of  d • • • ; , > i < >i \ making  a nd 

system  monitoring , for  several  reasons: 

(1)  Almost  all  systems  that  are  not  completely  automatic 
in  operation  and  self-monitoring  rely  significantly 
on  humans  to  detect  out-of-tolerance  conditions  and 
to  formulate  and/or  choose  among  alternatives  for 
correcting  those  conditions.  Many  systems,  in  fact, 
are  designed  around  the  notion  that  humans  are  better 
able  to  perform  such  functions  than  are  hardware  or 
software  mechanisms,  particularly  where  redundant 
procedures  for  monitoring  and  decision  making  can  be 
devised.  To  our  knowledge,  there  are  few  data  that 
either  affirm  or  disaffirm  that  notion,  and  no 
descriptive  model  of  group  performance  is  available 
on  which  to  base  a prediction  that  can  be  compared 
with  the  predictions  of  models  of  hardware  and  software 
performance . 


(2)  Prescriptive  models  of  group  decision-making  and 
monitoring  already  exist  in  the  literatures  of 
decision  theory  and  operations  research  and  can  be 
employed  as  yardsticks  against  which  to  measure 
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anticipate  in  more  than  a qualitative  way  the  :u- 

of  variables  such  as  leadership,  which  ar  > unices  1 \ 
associated  with  group  enterprise.  Empirical  data  and 
the  definition  and  formulation  of  predictive  models 
are  critical  here  if  future  operating  teams  and  crews 
are  to  bo  assembled  on  other  than  an  ad  hoc  basis. 

Advancing  The  State-Of-The-Art  with  Respect  to  S|  cific 
Model  1 ing  Ap p roac hes 

In  the  course  of  our  review  we  have  also  identified  specific 
emendations  that  would  further  the  development  of  particular 
Is  and  make  them  more  applicable  to  command  and  control 
: • ms.  Those  recommendations  are  1 isted  below. 

1 Data  Bank  and  Network  Approaches 
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4.4.2  Control  Theoretic  Models 

1.  Extend  optimal  control  attention-sharing  model  to 
apply  to  non-s tat  ionary  situations. 

; 

2.  Develop  methods  for  applying  manual  control  models  to 
mul ti-operator  situations . 

3.  Develop  more  formal  procedures  for  specifying  control 
and  decision  cost-functionals  for  the  optimal  control  model. 

4.  Develop  rigorous  schemes  for  estimating  statistical 
confidence  limits  on  the  parameters  of  the  optimal  control 
model.  Evaluate  and  extend  formal  parameter  optimization 
procedures . 

J 

i 5.  Extend  models  that  account  for  the  effects  of  work- 

induced  stress  on  control  and  monitoring  behavior. 

I 

6.  Extend  methods  for  integrating  the  representations 
of  attontional,  display- scannino,  and  control  workload. 

4.4.3  Human  Information  Processing  Mod» 

1.  Conduct  research  to  yield  a taxon'  my  that  relates, 
elemental  human  information  processes  to  human  ■ r'oi  line- 
requirements  in  realist  ic  systems  context:;.  D veloj  t.lieo: 
that  permits  task  requirements  to  ! > express,  .i  in  t . of 
component  information  processing  activities. 

2.  Exa;  im  tht  1 ink  a ■ ..  bet  w ••  dot  ect  ion  th  ••  a y mk; 
contro  ] - thoor  y-b.-  sod  decision  models  to  art  ieelatn  i singlt 

j model  applicable  to  dynamic  dec  i s i on-  niak  i ng  activiti'-s. 
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3.  Adapt  (■;'£•*  i : . • hn  : in  informal  ion  processing  stage  nn  1 Is 
and  apply  then,  to  t ' !••:>;  .cut  at  : o , of  pi  ict  i r 1 1 . i s. : - . 
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Data  Bank  Formulat ions 


NOTE ; Thouyh  not  "models"  .in  the  sense  in  which  that  term  is 
used  elsewhere  in  this  report,  human  performance  data 
banks  arc  of  demonstrated  utility  to  larger  efforts  aimed  at 
estimating  the  reliability  of  various  man-machine  configurations. 
Because  they  continue  to  be  of  value  to  designers  and  analysts 
and  because,  as  a group,  they  make  explicit  a large  set  of  opera- 
tor behaviors,  performance  criteria,  and  error  types  of  interest 
to  modelling  activities,  two  prominent  examples  of  the  data  bank 
approach  are  included  here. 


References : 


Payne,  D.  & Altman,  J.  W.  An  index  of  electronic  equipment 
operability:  Report  of  development.  American  Institute  for 

Research,  January  1962. 

Rigby,  L.  V.  The  Sandia  human  error  rate  bank  (SHERB) 

Human  Factors  Society  Symposium  - Los  Angeles  Chapter,  1967, 
(5-1)  - (5-13). 

Swain,  Alan  D.  Shortcuts  in  human  reliability  analysis. 

Taper  presented  at  NATO  Advanced  Study  Institute  on  Generic 
Techniques  in  Systems  Reliability  Accessment,  U.  of  Liverpool, 
England,  July  16-27,, 1973. 


D<  isc  ript  ion 


Data  bank  formulations  provide  a mechanism  for  prediction 
of  performance  in  prospective  systems  through  the  use  of  tabular 
arrays  containing  historical  laboratory  and  field  data  on  human 
reliability.  Though  the  formulations  differ  in  specific  details , 
all  attempt,  to  capture  and  represent  performance  in  subtask  units 
that  can  be  identified  .in  a variety  of  system  contexts  ( e . g . , 
component  inspect  ion , ma nu  il  operal  L<  n of  sw itches,  etc. ) . Most 
data  banks  are  concerned  with  at  1<  ist  two  aspect's  of  subtask 
performain  ■ : (I)  the  reliability  ( alternatively,  unreliability) 

with  which  a given  subtask  is.  performed  and  (2)  the  t i r . • taken  (on 
the  average)  to  complete  the  task..  A final  generalisation  is 
that  mod.  formulation.-,  contain,  ineldil  ion  to  tabulated  data,  a 
rationale  to.  the  comhina  I ion  ot  suld.ask  exeunt  ion  t i and/or 
: Mabilit  i a in  such  a way  that  lurjcr  par  fan:  :nce  aggregates 

cane  ily/.ed.  This  set  of  char  c!  er  i st  ics  maker,  the  data  banks 
very  si  ' 1 a r inruie.'pL  to  thox  toy  in  prod  i ot'  i on  o . hardware 
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reliability.  The  similarity  is  of  benefit  in  an  analysis  of 
systems  design  concepts  where  there  is  a desire  to  evaluate 
human  performance  in  the  same  terms  as  equipment  performance 
(e.g.,  failure  rate  per  n operations). 


Data  Store  (AIR) 


The  first  of  the  data  banks  to  be  formulated  and  one  that 
continues  to  be  of  utility  to  performance  prediction  is  that 
developed  by  AIR  (American  Institutes  for  Research)  in  196k. 
Called  "Data  Store,"  this  system  breaks  each  step  or  actio’  in- 
volved in  a procedure  into  three  factors:  (1)  nature  of  the 

stimulus  or  input  to  which  the  operator  must  respond  (e.g. , 
flashing  of  a light,  movement  of  a pointer  on  a scale,  etc.)  ; 

(2)  nature  of  the  perceptual  or  cognitive  mediating  processes 
required  (e.g.,  decision  making,  short-  or  long-term  memory, 
etc.),  and  (3),  nature  of  the  output  required  (e.g.,  movement 
of  a control,  speaking,  etc.).  Associated  with  each  of  these  is 
a measure  of  the  time  taken  to  respond  in  the  required  way  that 
has  been  derived  on  the  basis  of  laboratory  studies  and  available 
field  data,  along  with  an  estimate  of  the  rate  at  which  errors 
occur  in  that  activity.  Where  an  activity  can  be  parameterized 
and  the  precision  of  the  estimate  depends  on  the  values  of  the 


parameters,  "Data  Store"  provide 
activity  and  a "time  added"  for 


s a "base-time"  for  the  basic 
each  of  the  critical  sots  of 


values  on  each  parameter.  When  using  the  system,  the  designer 
first  chooses  the  activity  that  is  most  similar  to  that  he  is 
attempting  to  predict  and  then  adds  to  its  "base  time"  all  "time 
added"  increments  that  apply.  Aggregate  reliabilities  are 
handled  similarly,  except  that  reliability  figures  for  the 
critical  values  on  each  parameter  are  mul  t ipl  .ied  rather  than  summed , 


SHERD  (S. india) 

Since  "Data  Store,"  many  human  performance  data  banks  have 
been  developed.  Here,  we  shall  nvnt  ion  only  one  other,  the 
SHERD  (Sandia  Human  Error  Rate  Bank) , developed  and  used  in  con- 
nection with  THERP  (Technique  for  Human  Error  Rata'  Predict  ion) 
by  Rigby,  Swain  and  Rook. 

SHERD  is  a compilation  primarily  of  four  major  categoric:; 
of  error:  (1)  assembly  errors,  committed  in  the  production  c>' 

electronic  components  ind  oguipm  *:  ; (2)  install  it  ion  o:  tors, 

committed  during  Lnslal  I at  ion  and,  r intograt  ion  of  sm  '•  1 1 or 
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units  into  larger  units  (e.g.,  sub-assemblies  into  assemblies) 
of  equipment;  (3)  operator  errors,  committed  in  the  course  of 
operating,  transporting  or  handling  equipment;  and  (4)  errors 
committed  in  the  course  of  servicing  and  maintaining  equipment. 

Each  of  these  categories  is  further  divided  into  subcate- 
gories that  identify  more  or  less  exactly  the  task  of  interest, 
the  type  of  error  of  interest  and  the  criterion  for  assigning  the 
performance  to  a given  error  category.  Against  these  are  posted 
the  major  statistical  descriptors,  mean,  standard  deviation  (or 
other  available  measure  of  dispersion),  range  of  error  rates,  and 
(where  identifiable)  shape  of  distribution,  available  from 
laboratory  and  field  observations  relevant  to  that  performance- 
task-error  combination. 

To  aid  in  sensitive  exploitation  of  the  data  base , additional 
information  concerning  the  subject/operator  population,  work 
environment,  climate,  etc.  in  which  tabled  data  were  accumulated 
is  abstracted  and  maintained  as  part  of  the  data  base.  Certain 
dimensions  of  this  supporting  data,  thought  to  be  critical  to 
interpretation,  are  rated  on  a seven  point  scale  corresponding 
to  the  range  from  -3a  to  +3u  of  the  norma]  distribution  in  an 
effort  further  to  assess  their  influence  in  producing  observed 
results.  Finally,  evaluations  of  reliability,  validity,  gener- 
alizability  and  observer  credibility,  and  of  any  other  qual  i fying 
information  relevant  to  task,  nature  of  error,  situation,  etc. 
are  made. 


Like  Data  Store,  SilERB  can  be  used  to  supply  data  required 
for  network  analysis  or  for  assessment  of  single  task  performance. 
In  principle,  one  simply  socks  a match  between  the  activity 
(or  activities)  to  be  studied  and  the  keyword  and  descriptive 
structure  of  the  data  bank,  acqu ires  data  posted  against  relevant 
task-performance  p.  rameters,  uses  the  support  Lng  Lnformat  Lon  to 
achieve  a fine-grain  correspondence  b- tween  situational  conditions 
and  to  qualify  the  tabled  value,  and  makes  his  estimate. 


I npi  1 1 Para  net  ei  s 


Data  link  formulations  do 
the  normal  s;  use  of  the  term.  V.'h 
idea  of  tin.-  character  i st  ics  of  t 
so  that  appropriate  portions  of 


not  have  "Input  Parameters"  in 
at  is  required  is  a fairly  nroci 
he  prospect ive  t asks  and  sub  inks 
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Model  Outputs 

Model  outputs  are  typically  (1)  the  time  required  to  each 
member  of  a set  of  specifiable  subtasks  and  the  aggregate  task 
time;  (2)  the  probability  associated  with  success  (or  failure)  in 
the  execution  of  each  subtask  and  the  success  (or  failure)  of  the 
task  aggregate. 


Model  Validation 


Since  the  data  bases  of  the  various  formulations  are  con- 
stituted from  existing  laboratory  and  field  data,  they  are 
inherently  valid,  at  least  over  the  range  of  conditions  represented 
in  those  studies.  Their  validity  when  applied  to  new  situations 
is  limited  by  the  objective  similarity  of  the  prospective  tasks 
and  conditions  to  those  under  which  the  data  were  collected  in 
the  first  place,  as  well  as  by  the  sensitivity  of  the  user  in 
classifying  the  characteristics  of  a given  task  for  purposes  of 
relevant  data  base  information. 


Comments 


The  idea  of  tabulating  results  from  laboratory  and  field 
observations  and  of  using  these  to  estimate  performance  in  a 
system  yet  to  be  designed  is  clearly  attractive,  since  it  offers 
the  possibilities  of  great  economy  (over,  say,  simulation  methods) 
and  a standardized  methodology  for  reliability  analysis.  Asso- 
ciated with  all  data  banks  formulated  to  date,  however,  are  a 
number  of  major  problems.  One  is  the  fact  that  many  of  the  data 
contained  within  them  represent  empirical  outcomes  of  laboratory 
studies  whose  generality  may  be  suspect.  A second  is  the  fact 
that  much  artistry  may  be  required  to  match  tabled  descriptions 
of  the  tasks  against  what  may  be  only  vaguely  conceived  future 
tasks.  A third  is  the  likelihood  that  major  doubt  concerning 
the  operation  of  new  system  will  center  about  tasks  for  which 
there  is  no  directly  relevant  antecedent  experience,  hence  no 
counterpart  in  the  data  base  with  which  to  estimate  performance. 

Despite  these  problems,  the  data  banks  may  often  provide  ojh 
with  a ready  means  for  assigning  an  order  of  merit  to  each  memboi 
of  a set  of  design  alternatives,  and  aid  in  pinpointing  areas  of 
the  design  that  will  require  careful  consideration,  empirical 
study,  etc.  In  this  mode  of  use,  they  are  probably  indispensable 
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Siegel  & Wolf  Model  of  Perf  orrnan  ce  Under  St  re  s s 
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Description 

This  model,  originally  derived  for  use  by  the  authors  in 
their  simulation  technique,  is  based  on  the  Cannon-Bard  theory 
of  emotion  which  views  mild  stress  as  producing  a facilitative 
effect  on  performance  and  extreme  stress  as  producing  an 
inhibitory  effect. 

The  primary  assumption  of  the  model  is  that  "the  'certainty' 
in  the  operator's  'mind'  that  there  is  insufficient  time  remain- 
ing to  complete  all  [the]  essential  subtasks  when  performing  at 
normal  speed  and  efficiency  will  cause  a state  of  stress  on  the 
operator"  (p.17).  Two  parameters  are  critical:  (1)  the  average 

time  required  to  complete;  the  subtasks  in  question  as  compared 
to  the  time  available  to  >mplete  them,  and  (2)  the  "stress 
threshold,"  defined  as  that  value  of  stress  at  which  the  effect 
on  performance  changes  from  one  of  facilitation  to  one  of 
inhibition.  Stress  is  formally  related  to  time  as  follows: 


i’.  . E 

1 3 

Sij  = -T.  - TijU 


where  Tj  = the  total  time  available;  T j ^ - the  time  elapsed  up 

to  but  not  including  accomplish:"  >nt  of  subtask  i ; and']']  j''  - the 
average  time  required  for  completion  of  all  remaining  essent  ini 
subtasks,  assuming  no  failures. 

The  probability  of  successful  complct  ion  of  a subtask  in- 
creases linearly  with  Sp-j  from  a subtnsk  input  value  of  p]  j, 
which  represents  the  probability  that  the  average  operator,  j, 
can  perform  subtask  i successfully  while  not  under  stress,  unt  il 
a value  of  unity  is  reached  at  the  stress  threshold.  The 
probability  then  assumes  the  avom  :o  value  pj  ;,  after  which  it 
decreases  linearly  until  il  roach*.’.",  a value  equal  to  the  stress 


L 
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threshold  plus  unity.  At  this  point,  it  levels  off  at  a value 
that  is  decreased  from  py  j by  an  amount  equal  to  !-pj  j.  The 
following  equations  define  the  exact  probability  of  successful 
accomplishment  as  a function  of  Pi  , , S ^ and  the  stress  threshold, 

M . : ' j 

1 


, p.  . + 

I 13 


if  S . . < M . 
il  1 


Pii 


= / 


p.  . (S.  ,+1-M.)  + (M.-S.  .) 
13  11  1 1 11 


if  M.  — B.  . — M .+1 
1 il  1 


l 2 Pi-f1 


if  S.  . > M . + 1 
il  1 


These  exact  probabilities  are  mapped  onto  binary  success/failure 
outcomes  by  comparing  them  with  a pseudo-randomly  generated 
number  uniformly  distributed  in  the  unit  interval.  Success 
is  assumed  if  R3  is  less  than  p — ; otherwise,  failure  is  assumed, 
implying  that  over  many  trials,  there  will  be  a failure  with 
probability  p^ . . 

Input  Parameters 

Values  must  be  input  for  the  following:  T.  •I‘,  T-,T.  U,  M.  and 

Pi  j . As  employed  rn  the  Siegel  and  Wolf  simulation,  T-j  , -Tj  j,  and 

Mj  arise  out  of  running  computations  on  the  network  of  task’ times 

and  T.  are  derived  from  prior  analysis  of  task  performance  time 
il  ‘ 


Model  Output 

As  indicated  above,  the  outputs  of  the  model  are  (1)  t he 
exact  probability  that  a given  task,  i , imbedded  in  a sequence  of 
tasks,  [i],  w.il.l  be  performed  successfully;  and  {?.)  the  derived 
probability  (equal,  to  unity  or  zero)  that  the  task  wi  I I be 
accompl i shed . 
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Model  Validation 


The  model  has  been  validated  by  Siegel  and  Wolf  in  the 
course  of  their  simulations  of  a wide  variety  of  unitary  and  dual 
operator  tasks  and  seems  to  represent  the  observed  relationships 
between  stress  and  performance  quite  well.  Major  experimental 
emphasis  has  centered  on  the  search  for  appropriate  values  of  Mj 
for  any  given  simulation.  The  authors  report  that  original  values 
of  M.  for  any  given  simulation.  The  authors  report  that  original 
vcilu^s  of  this  parameter  were  based  on  a derivation  from  the 
Cannon-Bard  theory  but  that  these  have  been  modified  on  the  basis 
of  an  empirical  study  by  Siegel,  Wolf  and  Sorenson,  1962. 


Comment 


The  model  successfully  integrates  a nominal  concept  of  slack 
time  (as  employed,  for  example,  in  PERT)  with  the  reasonably  well 
validated  empirical  finding  of  an  inverted  U relationship  between 
stress  and  performance.  A particularly  desirable  characteristic 
for  complex  system  modelling  is  the  ability  to  generalize  the 
stress-performance  relation  to  n operators  performing  as  a team. 
It  is  likely  that  the  model  can  be  employed  outside  of  the  Siegel 
and  Wolf  simulation  with  a reasonable  promise  of  success  if  an 
adequate  basis  for  deriving  the  value  of  the  key  parameter,  Mj, 
can  be  found. 
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Siegel  & Wolf  Model  of  Subtask  Execution  Time  Under  Stress 


Re for once 

Siegel,  A.  I.  and  Wolf,  J.  J. 


Description 

This  model,  like  the  stress  model  described  earlier,  is 
employed  by  Siegel  and  Wolf  to  provide  input  into  a network 
simulation  model.  Its  purpose  is  to  provide  an  estimate  of  the 
time  required  by  an  operator  to  complete  a given  subtask  under 
stress  when  the  time  (£•_•)  required  by  the  average  operator  and 
the  average  standard  deviation  of  that  time  under  non-stressful 
conditions  are  known. 

The  model  uses  the  value  of  S^j  computed  as  described  in  our 
earlier  summary  and  the  value  M assumed  for  the  stress  threshold. 
Performance  is  then  given  as 


where  V.  . = t . . + K . . a .i  j . (K.  . is  a random  deviate,  o.  . is  the 

ID  i D i D ID  ID 

average  standard  deviation  of  performance  tire's,  and  Fj  is  the 
time  required  by  the  average  operator  to  perform  the  task  when 
nol  under  stress. 
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Input  Parameters 

Required  are  the  values  of  S -j_ -•  , t"i-j,  M j , K ^ •,  and  V j , 
former  three  of  which  are  computed  and/or  employed  in  the 
model  previously  identified.  Kj_  j is  a variable  generated 
basis  of  random  sampling  from  a normal  distribution. 


the 

stress 
on  the 


Model  Out put 

The  primary  output  of  the  model  is  an  estimate  of  the  time 
required  to  perform  the  task  under  various  levels  of  stress. 


Model  Valida t ion 

None  beyond  that  involved  in  normal  execution  of  the  Siegel 
and  Wolf  simulation  model. 


Comment s 

Comments  here  are  similar  to  those  made  in  connection  with 
the  S & W stress  model . The  model  is  modular  and  should  be 
useable  in  any  task  sequence  where  prior  empirical  data  regard- 
ing execution  times  and  stress  thresholds  arc  available.  A 
shortcoming  for  simulations  that  concentrate  on  "process"  rather 
than  taking  a black  box  approach  is  that  the  model  is  essentia]  ly 
apsychological . Beyond  the  fact  that  it  provides  for  the  repre- 
sentation of  inter-  and  intra-.in'divi  dual  differences,  it  is  not 
concerned  with  more  than  empirically  observed  relations  between 
stress  and  performance. 
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Preyss ' Bayesian  Model  of  Loarndnq  Behavior 


Preyss , . A . ' A Theory  a n 1 I iel  of  Human  Learning  ivi  r in 
Manual  lontro  Tas  . Sc  .D.  Dissertation,  ■ . I . T . , 
if  7 • 77  ■ ’ M--;. - ; I'to  7 . 


Preyss,  A.  El  and  Meiry,  J.  i. . “Stochastic  modeling  of  hu:  an 

learning  behavior,"  Third  Annual  NASA-Uni versify  Conference 
on  Manual  Control,  NASA  SP-144,  1967. 

Meiry,  J.  7.  "S'  ochasti  : modeling  of  human  learning  b :havi  r,” 

Fourth  Annual  NASA-University  Conference  on  Manual  Control, 
NASA  3P-192,  1968. 


T.ais  model  attempts  to  account  for  the  behavior  of  naive 
subjects  ir.  a time-optimal  stat  ? regulation  task  by  rr  ;ans  of 
Eayesiar,  : les:  ' hypothesis  testing.  The  context  in  which  the 

model  was  first  devs  1 >t  1 was  bang-bang  contr  1 of  a pur<  ' ertia 
' ant  = + U) . The  model  ..was  later  extended  t > covi  ••  < ier  secoi  }- 

rder  . . - , b o n . - lb  ( x t k x — t U ) and  uns  tat  ( x — kx  = 4*  I ) . 

The  me  iel  assumes  that  the  human  controller  for:  ..',  an  Maternal 
model  of  the  system  state  spac , divided  into  a coat-.  • grid , and 
that  for  each  point  on  the  grid,  he  formulates  hypotheses  about 
the  results  of  a control  input  switch  at  each  point.  Observing 
the  system’s  behavior,  he  generates  control  decisions,  ■ iceeutes 
them,  obs  'ves  their  results , and  uj  a es  his  hypotheses  using 


king  Newell , Sha w , at  3 Simon’s  dietu  that  ”at  • ana- 
: . obse  id  behavi  >r  if  an  org  riism  ; s provid  i by  a 


progra 
: i ha  v ' 
accord' 


: 1 . tuning  ' - model  j - am 
milar  t those  ‘naive  sub.je 


f pri mit ive  inf  rmation  j • ■ oses  thal  gem  *ates  his 
Preyss  impler  >nted  a c j . ?r  pi  igrai  struc  ■ . in 
with  the  model . Tuning  tl  model  parnmel  *s  1 ‘odu 
i3?3  similar  t those  naive  subjects,  suc- 
■ idu  • "Mi. : a program  ‘ -hat  no4  only  c w wp  i :.o  ‘ • 

■ - ' . ■ itra  • ".  , but  als  lid  so  i . such  ; 1 its 

res:  '•  j :::-v  :b  u 4 i.ur.  io:  ' 1 7 ii  ; ! 4 ' : 1 : . i i . ■ . 1 • • 

.-','<’7  :d  cf’  h'J".'  ' li-M  ' 4 
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Model  Outputs 

Tht ■ model , with  suff ici en 
of  control  3 nput s stat  1 st : ;all y 
naive  human  subjects  ns  they  re 
convi  rge  on  the  >pt3 mum. 


in i ng , : an  reproduc  s ; u e n c e s 
similar  to  thos<  exhib i ed  by 
’ine  i h<  Lr  • ntrol  strategies  to 


M< (del  Validal  ! >n 


The  model  has  proved  sufficiently  robust  to  be  able  to  mimic 
human  performance  with  three  different  sets  of  controlled  dynamics. 
In  each  case,  however,  model  parameters  had  to  be  adjusted  to 
match  the  human  subjects'  data,  a fact  that  severely  limits  the 
use  of  this  model  in  a predictive  role. 


Comments 


This  mod^l  is  noteworthy  because  it  represents  one  of  the 
few  attempts  undertaken  to  account  for  the  learning  behavior  of 
naive  controllers  in  a state-regulation  task.  As  a descriptive 
model,  it  suffers  from  the  need  to  match  several  context-specif ic 
parameters.  It  is  unlikely  to  prove  very  useful  for  a j • 1 ri 
predictions  of  human  behavior. 


Bolt  Reran,  k 
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Do  s c:  r lot  ion 

The  "crossover"  model  is  used  here  as  a rubric  for  the  class 
of  models  that  treat  the  human  controller  by  methods  of  quasi- 
line. ar  describing  function  theory.  The  quasilinear  model  of  the 
human  operator  consists  of  a describing  function  that  accounts 
for  the  portion  of  the  human  controller's  output  that,  is  linearly 
related  to  his  input  and  a "remnant"  term  that  represents  the 
difference  between  the  output  of  the  describing  function  and 
that  of  the  human  controller. 


The  linear  describing  function  portion  of  the  quasilinear 
model  takes  on  various  forms  depending  on  the  precision  with 
which  one  attempts  to  reproduce  th  • human  controller 1 s charac 
teristics.  A fairly  large  1.  > ly  of  dal  t can  be  accounted  for 
by  a model  of  the  form  (McRuer,  Graham  et  at.,  1965). 


Yp(j«,) 


DwT  + 1 

J j 

) 'V,  I ! 


I 


l I 


Kq  a t i ■/. a t i on  I . i 


a t.  i ons 


9 hi s describing  function  cot  crises  faclors  related  to  some 
human  limitations,  namely,  react  in  d ■ I ays  ( « ) arid  1 iqs  attribute 

to  t ! . • net  . ■ t 1 : sysl  ('  : ) lot  factor  i ;od  i < > i >del  the 

hut.:  m ' s«  ad  : • i ve  cqu  il  i y.al  ion  cli  a » 'to  i .tic:.. 
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The  mos i nor tan t , and  . ■ ' e I < ■ ■ j a: . f , result  of 

quasi  linear  m tnuul  control  theory  i;.  embodied  in  Hi- . "crossovaj 
model"  which  relates  yu(j  ) to  the  trnnsfei  funct ion  of  the 
controlled  element  yt.  (j.-.  ) by  th-  equation. 


O' 


yp(  im) 


yc  ( j,j) 


c;  < 


- in.  r 


;jo 


where  m is  the?  crossover  frequency  ana  x the  elfectave  tim  ■ 
del  ay . 

The  situation  with  respect  to  th.  ■ remnant,  portion  of  the 
quasilinear  models  is  less  well  developed.  The  current  view  of 
remnant  in  quasi lineal  manual  control  theory  is  that,  in  the 
absence  of  display  scanning,  remnant  .is  due  largely  to 
irreducible  stochastic  variation  in  the  human  operator 
(McRuct  and  Krendel , 1974,  j. . 69).  He:. nun!  in  not  error  in 
modeling  the  deterministic  portion  of  the  controllei 1 s response, 
although  such  errors  could  contribute  to  remnant.  Models  for 
single-loop  remnant  consist  of  empirically  obtained  f irst-orde  r 
noise  spectra  injected  at  the  operator's  input  (McRuer  and  K rondel 
1974,  p.  34) . Fairly  elaborate  models  for  multi-display 
scanning  have  been  developed  and  have  been  used  to  predict, 
remnant  in  multi-loop  situations  (see,  e.g.,  Allen,  Clement  and 
J e >:  t 19  70). 


Input  Parameters 

As  with  other  control  and  decision  theoretic  models,  this  one 
assum.es  precise  knowledge  of  the  transfer  funct.i  >n  of  the 
controlled  element  and  of  the  disturbance  inputs.  The  controller' 
dose ribing  funct ion  is  determii  a as  to  form  and  pa  u ;ters  via 
adjustment  rules  that  have  been  determined  from  theoretical  consi- 
derations concerning  close  d -1  oo;  pi  r forma nee  and  an  t xt  o.nsi  \v 
empirical  delta  base . Broadly  : unking , the  fori  oi  the  describing 
funct  ion  is  chos-n  t o give  goo-1  low  frequency  r-  por.se  and  abs.ol  lit 
stabil  ify.  Tiie  p : - a ureter  v<t  1 u-  car.  be  selected  according  to  sou. 
optimizing  criterion;  how  ve r,  'a.,  bird  practice  liar.  1?  n to  input 

Vi : I i 1 1 : 5 foi  < l 1 1 . 1 i : SUC'h  , ( 1 l ’l  U dal  ) . 
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Mo del  y c 1 1 1 d a t i o n 

The  basic  "crossover  model"  has  be-. -a  validated  for  a wide 
variety  of  single-input,  single-output  systems.  Accordingly, 
it  has  been  used  in  a numerous  practical  applications  (see 
Me Rue r and  K rondel,  1974).  The  model  has  also  been  applied 
and  validated  in  more  complex  tusks , but  to  a.  significantly 
lesser  extent. 


Comments 


Despite  their  unquestioned  success  in  the  analysis  of  a 
number  of  significant  pilot-vehicle  problems,  qua  si. -linear 


models  do  suffer  certain  deficiencies,  especially  with  respect, 
to  more  complex  tasks.  One  problem  concerns  the  extension  of 
the-  concepts  developed  for  single-loo;  control  situations  to 
more  complicated  multi-input,  multi-output  systems.  Another 
problem  lies  in  selecting  the  parameters  of  the  pilot  describing 
function.  These  parameters  are  task-dependent  and  have  been 
selected  traditionally  on  the  basis  of  verbal  rules.  The  ability 


to  predict  remnant 


complex  situations  within  this  framework 


is  still  inadequate.  Because  remnant  can  account  for  a signi- 
ficant fraction  of  the  controller's  output  and  increases  with 


task  complexity  (McRuer,  et  al. 
dieting  remnant  is  required  to 
Finally,  quasilinear  models  are 
stationarity  and,  further,  are 
random  inputs.  Thus,  there  is 
for  extrapolating  these  mode! s 
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to  problems  inv, 
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, 1965),  a good  model  for  pre- 
fa  r •’  performance  accui a 1 ..-1  \ . 

b...-  .1  on  assumptions  of 
al;;.  !.  'wholly  developed  for 

no  theoretical  justification 
to  time-varying,  non-stat ionary 
vine  non-random  inputs.  This  i s 
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Description 

"Paper  Pilot"  is  a fixed-form,  parameter  optimization  model 
of  the  human  controller.  The  model  was  developed  to  predict 
"pile!  ratings"  for  vehicles  and  tasks  not  believed  to  be  covered 
by  existing  MIL- Spec  flying  qualities . The  i lo]  Lncorpor at  ;s 
a pile -.-vehicle  model  with  a form  that  is  similar  to  those  need  in 
describing-function  analysis.  However,  the  distinctive  featur  of 
paper  pilot  is  that  the  parameters  of  the  model  for  the  pilot  ire 
selected  to  minimize  a "rating  cost"  functional  of  the  form 

a- 

R = 1 + K ( --)  + >:.  K T (1) 

o J l L . L ■ 


where  R is  the  predicted  Cooper  rating,  o is  sour*  measure  of  over- 
all system  performance  (say,  a linear  combination  of  rms  varia- 
tions in  flight  path  and  aft  i trade  vu‘  iublos,  V|  is  the  lea  . t in 
constant  generated  by  the  pilot  in  the  i * roin  rol  lo  p,  and  K 

and  Kj  . arc  weichbi  nc;  coo f f Lei  nts.  Th  • variable  ^ 

the  desired  ; rrforr:  me  - level  in  a particular  task.  System  per- 
formance degrades  the  rat  i ng  only  wh  -n  o o . 

The  rating  ex;  ession  inco  a t juanti ties  re]  I ! t b 
performance  ml  workload,  wher  workload  i dofin  ><1  in  tm  • ..  o'  th  * 
pilot  leal  time  constant  • in  the  varimw  loop-  (aa  i nt  orpre : a!  i on 
that  i 1 l on  jrossover-mo  i > \ ■ ) . 1 *unt  that  < 

per foi  i : ! ••  ■ : i i > i : ran  c i r 1 i to  ' i I t i 

cons  fra  i rvd  . Th  i • prompt  s t he  i m t"  e ' o • i a t i on , g i von  by  Di  1 l 

, that  ’ ■ 1 i • 1 ■ ■ t . ■ i a 1 i it 


combination  of  workload  and  performance,  and  "washes  out."  exocf  1- 
incj]  y poor  or  good  porfornnaco  in  determining  tho  rating.  Init- 
ially "Paper  Pilot"  war.  applied  to  a VTOL  hover  task.  Since  then 
it  has  been  used  to  analyze  a variety  of  aircraft  control  tasks. 


Model  Inputs 

The  mathematical  descriptions  of  vehicle  dynamics,  gust  dis- 
turbances, etc.,  necessary  for  a control  system  analysis  must  be 
provided  as  with  all  quantitative  manual  control  models.  In 
addition,  one  must  specify  tho  for:-:  of  pilot  model  (.loop  closures 
and  equalization  characteristics).  Further,  the  weighting  coef- 
ficients in  the  rating  cost  functional  must  be  provided. 


Mode 1 Outputs 

Model  outputs  are  predicted  pilot  rating  (on  a Cooper  or 
Cooper-Harper  scale),  parameter  values  of  the  pilot  model  and  per- 
formance measures  (i.e.  the  standard  deviations  of  all  motion 
variables) . 


Validation 

As  indicated  above,  Paper  Pilot  has  been  used  to  analyze  a 
variety  of  tasks.  Where  data  have  been  available;  to  validate  tho 
model,  the  results  have  been  "mixed."  In  general,  tho  model  pre- 
dicts pilot  ratings  with  reasonable  accuracy  but  does  not  do  so 
well  in  predicting  performance  scores. 


Comments 

The  "Paper  Pilot"  model  with  parameters  selected  to  optimize 
a rating  cost  functional  has  been  ippliod  wi  th  some  succor- 5; . This 
optimizing  approach  replaces  the  verbal  adjustment  rules,  of  des- 
cribing function  theory  with  a systematic  procedure  that  should  be 
inherently  more  "predictive."  However,  there  arc'  problems  and 
limitations  associated  with  the  a;  .roach  that  have  yet  to  be  re 
sol vod . 

As  no  "ad  above,  th'  model  do  03  not  prediet  perfornnnee  as 
v/- » I 1 as  desirable  (or  nohi  -voble  wi  th  other  mod, -Is).  An  important 
reason  is  that  exist  i ng  i ;-•  > 1 no.  ,nt  ions,  or  "Paper  Pilot"  mod  Is. 
do  not  i ir-oruorati  • pilot  to::. aunt.  Inclusion  of  a remnant  model 
may  not:  b-  • strai  ;h?  forward  i na'.nnu:  "a  ns  inherent  pi  lot  rfltidoi  , 
got  s con  found,  a!  by  "on  'doling  ■ rro-  ' in  !':■:•  1 • form  mo.  he  1 s.  such  us 
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j Another  problem  is  the  selection  of  weighting  coefficients 

in  Equation  (1).  Although  the  concept  of  relating  pilot  opinion 
to  measures  of  system  performance  and  workload  seems  sound,  gen- 
era] rules  for  choosing  the  K0  and  coefficients  will  be 

i needed  if  the  model  is  to  be  truly  predictive. 

I 

Perhaps  the  most  serious  drawbacks  to  this  technique  will 
arise  when  addressing  multi-output,  multi-axis,  multi-control 
problems.  Then,  as  in  the  describing  function  approach,  it  will 
be  necessary  to  postulate  possible  loop  structures  and  the  model 
forms  for  each  loop.  This  will  complicate  the  problem  of  choos- 
ing a rating  cost  functional,  increase  the  possibilities  of  model- 
ing error,  and  increase  the  number  of  parameters  to  be  optimized. 
These  factors  will  undoubtedly  jeopardize  the  predictive  capabil- 
ity of  the  techniques  and  will  also  magnify  computational  prob- 
lems. The  problems  associated  with  computing  the  optimizing 
parameters  are  far  from  trivial  even  in  a single  loop  case;  for 
multi-loop  systems  - where  each  loop  can  contain  several  param- 
eters - computer  time  could  be  excessive. 
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Description 

The  optimal  control  model  is  a stochastic,  time-domain  model 
for  the  human  operator.  It  ine  Lucas  a model  for  predicting  the 
random  component  of  human  response  and  is  not  limited  to  stationary 
control  situations.  It  is  capable  of  treating  multivariable 
systems,  as  well  as  si regie- lo  p systems,  within  a single  con- 
ceptual framework  using  state-space-  techniques.  The  basic 
assumption  underlying  this  approach  to  modeling  the  human 
operator  is  that  a highly- tra ; nod  human  controller  will  act 
in  a near-optimal  manner,  subject  t cer I tin  internal  constraints 
that  limit  the  range  of  his  b havi . • and  ilso  subject  to  the 
extent  to  wh'-'h  hr>  under;:  I and  • th'  >b  joci  ; of  t he  ta::1  . 
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It  is  convenient  t.o  consider  the  model  for  the  operator 
as  comprising  t'.-  follow  i ny : (i)  an  "equivalent" 

p •rceptual  mode  l that  train;  Later,  displayed  variables 
into  noisy,  delayed  perceived  variable:;  denoted  by  y;j(t).  A 
threshold  is  al  .•  > cos  si  lered  part  o the  perceptual  process  but 
may  actually  be  used  to  mode  1 t he  o : focts  of  non-ideal ized  display 
The  threshold  is  treated  bj  st  ti stical  l ineariz  tion  techniques ; 

( i i ) an  inf  on  it  ion  pi  icessing  model  that  attempts  to  estimate 
the  system  st  ifo  fror  th  perceived  d it  i.  The  in for mat  ion 
processor  consists  o ! opt  i i 1 estimator  and  predictor  and  it 
generates  the  minimum  -v  : i i ant’-  • estimate  >:(t)  of  x ( t ) ; (iii  ) a set 
of  "optimal  gains,"  1.*,  chosen  to  minimize  a quadratic  cost 
functional  that  express  s task  re:jui  remen Ls  (Kleinman  and  Baron, 
1970);  and  (iv)  an  equivalent  "motor"  or  output  model  that 
accounts  for  "bandwidth"  limitations  (frequently  associated  w.i  th 
neuromotor  dynamics)  of  the  .uinun  and  his  inability  to  generate 
noise-frcc  control  inpus. 

The  time-delay,  obsorvat  Lon-  and  inotor-no i ses  and  the  neuro- 
motor-lag  matrix  account  for  inherent  limitations  on  human  pro- 
cessing and  perceptual-motor  activity.  Methods  for  choosing 
values  for  these  quant  it  s have  been  determined  by  matching 
experimental  data  and  > valin  s have  b<  en  found  to  be  generally 

independent  of  t isk  parameters . The  obs<  rvai ion  noise  is  a key 
feature  of  the  model.  i;  is,  essentially,  a lumped  representation 
of  human  randomness.  From  the  standpoint  of  classical  quasi - 
lineai  describing  function  theory,  the  observation  noise  may  be 
thought  of  as  a model  for  remnant.  On  the  basis  of  considerable 
experimentation,  a relatively  simple  set  of  rules  for  predicting 
operator  remnant  has  been  found.  Specifically,  each  display 
variable  utilized  b\  the  operator  is  assumed  to  be  perturbed  by 
a white,  gaussian,  zero-mean  noise  process  that  is  linearly 
uncorrelated  with  other  noises  and  with  system  inputs  and  which 
has  a power  density  level  th.it  scales  with  the  mean  squared 
level  of  the  display  variable. 

Th  • optimal  est  i.mat  >r,  predictor  and  gain  mat  l i x repr<  sent 
the  sot  of  "adjustments"  or  "adaptat  ions"  by  which  the  human 
attempts  to  optimize  his  behavior.  The  general  expressions  foi 
those  model  elements  depend  on  the  system  and  task  and  are 
determined  by  solving  an  appropriate  opti.niznt  ion  problem,  and 
therefore , according  tc»  wel  I -defined  rule:;. 


M(  idol  In;  i * : i 


The  mo ile  1 inputs  in  dude  re;  resented  ions  of  system  dynamic:., 
t he  d i s p lay  :;ys*.  : , env  I ronmou  ' a 1 d i s t u b . see  <•«  mds,  t ask 

requ  i i lUirMit  s , a;  : the  ; 1 1 ai  • t -is.  spec  i f y i n i o{  • i . t ■ r 1 imi  t at.  ion;; . 
Tii.  system  dynamics  c>  ■■  rise  the  1 i near  i zed  civ  ■ i os  of 
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the  controlled  element  and  any  dynamics  associated  with  measure- 
ment, control,  and  display  systems  (also  linearized).  The 
equations  for  these  dynamics  are  expressed  in  s tate-variabl  _• 
form.  The  quantities  displayed  to  the  human  operator  are 
assumed  to  be  generated  by  linear  operations  on  the  state-  and 
control-vectors.  System  and/or  display  may  change  with  time, 
in  either  continuous  or  abrupt  fashion. 

Disturbance  and  command  inputs  must  also  be  specified. 
Disturbances  can  include . random  gusts  and  quasi-deterministic 
inputs  such  as  steady  winds-  and  wind-shears.  Commands  can 
include  tracking  inputs  to  the  pilot  that  are  predictable  in 
nature  but  not  known  preciseTy  by  the  pilot. 

Task  requirements  are  stated  in  terms  of  "cost  weightings" 
associated  with  system  variables  in  a quadratic  cost  functional. 

It  is  assumed  that  the  operator  selects  his  response  to  minimize 
this  cost  functional.  The  selection  of  cost  functional  weightings 
may  bo  based  on  objective  or  subjective  factors.  For  simple, 
single- variable  control  situations,  good  results  have  been 
obtained  with  a cost  functional  consisting  of  a weighted  sum  of 
system  error  variance  plus  control-rate  variance.  For  multi- 
input,  multi-output  systems,  coefficients  are  determined  from 
system  specifications  and  a knowledge  of  operator  preferences. 

Parameters  related  to  pilot  limitations  are  time-delay, 
observation-  and  motor-noise  ratios  and  a "neuromotor"  time- 
constant  that  reflects  a limitation  on  operator  bandwidth. 

Values  for  these  parameters  have  been  found  to  be  relatively 
independent  of  task  parameters. 


Model  Outputs 

The  solution  to  the  defined  optimization  problem  yields 
predictions  of  the  complete  closed-loop  performance  statistics 
of  the  system.  Probability  densities  of  all  system  variable.; 
(states,  outputs,  controls)  are  generated  as  functions  ol  t.  i 
along  with  mean  and  rms  error  deviations  from  the  nominal  flight 
path.  Moreover,  the  densities  of  the  pilot's  estimates  and 
estimation  errors  are  also  predicted  as  functions  of  time. 

All  computations  are  performed  using  covariance  propagation 
methods,  thus  avoiding  costly  Monte-Carlo  simulat  ions. 

However,  if  desired,  a "sample"  or  si.vu’  ; cion  version  of  the 
model  is  possible.  Given  the  probability  density  functions  for 
the  state,  it  is  possible  to  compute  ;■  ■ - . . i nq  f u 1 not  fo  • nance 
statistics  (e.g.,  one  can  compute  He  ;a  inability  of  a missed 
approach)  . It  is  i tnj  lortant  to  no  I e the  t h - ■ pi  1 i c t . 'd  p rob  d>  i 1 i t v 
den si  I i es  aio  condi t ioned  on  the  particular  choices  fot  sysla: 
parameters . Thus,  by  chang  i ng  part  c : system  vat  i able.*? 

between  computet  runs  and  obsot  v i ng  t ci  )•••  in  pet  forma”  -e, 

the  of  feats  on  per  fori  mice  of  chanm.  : :.  my  system  quant  ity  can 
b1  i fives  t in  ■ i • I . y . i t i'il  ly  and  r •.  1 . • vel  y u ■ - pet  - i ve  1 y . 
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The  model  has  been  subjected  to  extensive  validation  with 
very  encouraging  results.  Tt  has  been  validated  in  relatively 
simple,  stationary  control  tasks  and  in  more  coi  pi ex  tasks,  both 
stationary  and  non-s tat  ionary . Further  validation  is  needed 
for  applications  that  involve  pre-programmed  maneuvers. 


Comments 


The  optimal  control  model  is  very  general  as  far  as  manual 
control  models  go  because  of  its  time-domain,  state-space  formu- 
lation, its  normative  nature  and  its  explicit  information  pro- 
cessing sub-model.  These  features  have  allowed  models  for  task 
interference  (Abstract  Mo.  19),  scanning  (Abstract  ho.  16), 
decision  making  (Abstract  No.  17)  and  failure  detection  and 
identification  (Abstract  No.  20)  to  be  postulated  and  incorporated 
within  the  same  framework. 

Though  the  model  has  been  quite  successful,  there  remain 
several  problem  areas  with  respect  to  its  use.  The  model  is 
limited  to  linearized  analysis,  except  to  the  extent  that  memory- 
less nonlinearities  can  be  treated  by  statistical  linearization. 
The  choice  of  weightings  in  the  cost  functional  is  an  arc 
involving  the  judgement  of  the  analyst.  It  is  not  clear  that 
operators,  even  whan  trained,  will  adopt  the  same  or  similar 
criteria.  The  model  is  based  on  an  assumed  state  of  advanced 
training  and  high  motivation  and  its  value  in  dealing  with 
operators  who  do  not  fulfill  these  assumptions.  Finally,  no 
work  has  been  done  on  applying  t.hese  techniques  to  multi- 
operator  situations. 
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Descript ion 

This  model  is  designed  to  yield  highly  detailed  predictions 
of  the  human  operator's  control  behavior  in  a wide  range  of 
time-invariant  tracking  tasks,  including  both  pursuit  and 
compensatory  tracking  tasks  with  and  without  prediction  displays. 

The  modal  assumes  that  the  human  operator  perceives  the 
tracking  error  and  its  first  derivative,  his  own  control  output 
and  its  first  derivative,  plus  (for-  pursuit  tracking)  the  forcing 
function  and  its  first  derivative,  and  (for  predictor  displays) 
the  predicted  error  an  1 its  first  derivative.  Each  of  these 
signals  is  fed  both  to  a control  signal  generating  loo;.'  and  to 
a nonlinear  decision  algorithm  loop.  In  each  case,  the  signal 
vector  is  multiplied  by  a weighting  vector.  Since  the  weighting 
vector  can  be  different  for  the  two  loops,  a total  of  16  weighting 
parameters  must  be  specified. 

The  nonlinear  decision  algorithm  loop  incorporates  threshold 
elements  that  can  cause  the  control  signal  generating  loop  to 
hold  a particular  control  stick  position  for  a period  of  time, 
regardless  of  changes  in  the  other  signals.  Parameters  ol  the 
decision  algorithm  loop  include  a threshold  value,  a sampling 
Interval , and  a pa  ametet  that  comput  is  the  mml  it  ol  sampling 
intervals  over  which  a constant  control  output  should  be  held. 

In  all,  there  are  19  param  'tors  thit  must  be  sped  find  for 
the  model.  The  procedure  employed  for  dot  on  i n i :ig  them'  para- 
meters .involves  both  open-  and  closed-loop  optimizations  cat  t i .•,] 
out  using  input , output,  and  predict-'  d output  s i rials  record*  *d 
during  si  mint  ion  experiments . 
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f Iodol  Outpu is 

The  model  yields  predicted  time  histories  of  the  h;r:n 
operator's  control  behavior  for  a given  input  signal. 


Val  ida  tio.n 


Parameter  values  were  determined  for  a series  of  experi- 
mental conditions  using  four  different  sets  of  controlled 
dynamics  and  displays: 

(1)  II  (s)  = y — --- pursuit  tracking,  predictor  display 

s ( 1 + 0 . 4 s ) 


(2)  H (s)  = ~2- 
s 


pursuit  tracking,  non-predictor 
di splay 


(3)  K(s) 


1 .6 

s (1+0. 4s) 


compensatory  tracking,  predictor 
di splay 


(4)  H ( s ) - 1.6 


compensatory  tracking,  non-predictor 
di splay 


After  parameter  optimization,  the  control  output  signals  were 
matched  with  those  produced  by  the  human  subjects.  The  control 
outputs  produced  by  the  nonlinear  multi par an  a te  r model  matched 
those  produced  by  human  subjects  bettor  than  th  ■ control 
outputs  produced  by  a linear,  descri  bing-fur.eti  on  model. 


Comments 

This  model  is  noteworthy  for  its  ambitious  all-  • : 
match  the  de ‘ lils  of  human  control  behavior.  As  i , 
t :o  > ! , its  us  seems  l imi ted . Parametei  valu  s 

only  for  a types  of  systems,  and  lit  1 1<  quid  i i I ibio 

for  estimating  the  values  1 hat.*  would  arise  with  otlno  . • ' 'll  >d 

dynu:  ■ Lcs . 
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Description 

This  model  addresses  the  same  problem  as  the  Elkind-Miller 
adaptive  response  model.  It  assumes  that  following  a sudden 
change  in  controlled  process  dynamics,  the  human  controller 
proceeds  through  four  phases  of  activities: 

(1)  a retention  phase,  in  which  he  retains  his  pre- 
failure transfer  function, 

(2)  a detection  phase, 

(3)  an  identification  and  modification  phase,  and 

(4)  a steady-state  tracking  phase  using  the  appropriate 
post-failure  transfer  function. 

The  primary  differences  between  this  model  and  the  Elkind-Miller 
model  arise  from  the  use  of  phase-plane  boundaries  to  predict  both 
detection  and  modification  activities. 


Specifically,  the  model  reports  that  a change  in  dynamics  has 
occurred  whenever  the  system  error  and  error  rate  are  such  that 
the  system  state  exceeds  the  bounds  of  a particular  region 
surrounding  the  origin.  If  this  occurs,  the  model  adopts  a 
now  transfer  function  appropriate  to  one  of  the  alternative, 
"failed"  dynamics.  If  the  system  state  continues  to  dive1  jo  and 
crosses  a second  boundary  in  th  > phase  plane,  Hie  model  shi  I Is  l.o 
a second  alternative,  and  so  on. 

From  the  description  o r the  model  published  in  the  literature, 
it  .is  not  clear  on  what  basis  the  o:  :er  of  the  alternative  i • > i - 1 
s chosen . The  specific  oi  ior  used  Is  tha  oi  decreasii  j , >- 
bability , ■ ■ , the  most  probabl<  Llure  as  first,  thi 

the  next  i | . , etc . The  < -der  us<  5 also  hap]  ins  t > 1 

that  of  increasing  lead-gone rn t. ion  >‘o  jui  foments  on  the  hut. in 
c m t rol  i o',  ; n wh  e ••  H ■ " •->  " ••  • i • a a sur"  • i fir:;'. 
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In  either  case,  the  phase-plane  boundaries  app  ar  to  have  be  sn 
constructed  on  an  ad-hoc  basis  to  match  the  controller's  behavior. 

No  rules  are  given  for  constructing  these  boundaries  in  other  cases. 


Inpu ! P<  i ra  me  te  rs 


The  model  requires  the  transfer  functions  of  the  unfailed 
system  and  of  each  of  its  alternative  failed  modes.  The  statistics 
of  the  input  disturbance  are  also  required,  along  with  the  para- 
meters of  the  phase-plane  boundaries  that  trigger  the  mode-switching 
behavior  of  the  model. 


Mode 1 Outpu ts 

When  the  phase-plane  boundaries  are  established,  the  model 
yields  detailed  time  histories  of  system  control  inputs  and  system 
state . 


Validcit  ion 

The  model  was  constructed  in  the  context  of  failures  in 
stability  augmentation  for  the  roll  axis  of  VTOI.  aircraft  in 
hover.  The  model  was  shown  to  reproduce  the  qualitative  features 
of  a human  pilot's  responses  fairly  accurately.  No  attempts  to 
extend  the  model  to  other  contexts  were  reported. 


Comments. 

This  model  is  noteworthy  for  its  use  of  phase-plane  trajectories 
and  boundaries  for  the  detection  and  identification  of  changes  in 
controlled  process  dynamics.  This  approach  could  be  useful  in 
analyzing  failure  identification  tasks  for  which  data  have  already 
been  recorded,  but  without  specific  rules  for  i den i i ying  phase- 
plane  boundaries,  the  model  cannot  be  used  predict ively . 
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Description 

This  model  addresses  a specific  class  of  manual  control 
problems  in  which  the  controlled  process  exhibits  very  large 
time  constants.  The  specific  context  of  the  model  is  the  helms- 
man's steering  control  of  a supertanker,  the  dynamics  of  which 
may  be  described  by  the  equation 


Ts$(t) 
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+ a1v(t)  + a0 1 if»( t ); 
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I v ( t ) 
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where  ip  is  the  ship’s  heading  and  6 is  the  rudder  deflection  angl 
For  large,  loaded  supertankers,  the  value  of  T3  can  approach  five 
minutes. 

The  model  assumes  that-  the  he  Inman 
employs  an  internal  estimator  for  deducing  the  system  state 
variables,  and  an  internal,  simplified  model  of  system  dynamics. 
It  was  found  that  an  internal  model  of  the  form 
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either  model  could  adequately  reproduce  the  path  of  the  ship, 
but  that  the  non-linear  model  more  closely  approximated  the 
helmsman's  control  inputs. 


Input  parameters 


A mathematical  description  of  the  controlled  process  dynam- 
ics is  required.  If  an  accurate  fit  to  the  helmsman's  control 
inputs  is  desired,  sample  time  histories  are  also  required  in 
order  to  fit  the  parameters  of  the  non-linear  decision  model. 


Model  output 

The  model  can  be  used  in  simulation  exercises  to  determine 
the  ship's  stability  and  probable  course  deviations  during  vari- 
ous maneuvers. 


Model  validation 


In  addition  to  several  experiments  conducted  using  realis- 
tic simulators,  full-scale  experiments  were  run  on  an  actual  ship 
under  various  conditions.  Results  showed  that  in  each  case,  the 
helmsman's  internal  model  exhibited  a time  constant  very  close  to 
that  of  the  shio  itself. 


Comments 


This  model  provides  a valuable  case  history  that  should  bo 
reviewed  in  any  attempt  to  model  human  control  of  a system  with 
long  time  constants.  In  very  low-frequency  regimes,  the  cross- 
over model  does  not  yield  an  adequate  characterization  of  the 
controller's  control  behavior,  and  non-linear  decision  models 
must  be  employed. 
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Onstott's  Multi-Axis  Tracking  Model 
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Description 

This  model  is  designed  to  be  used  in  direct  digital 
simulation  of  pilot-vehicle  dynamics.  The  model  incorporates 
standard  crossover  models  of  pilot  dynamics  for  each  control 
axis,  with  inner-  and  outer-loop  control  of  appropriate  state 
variables.  In  addition,  however,  an  "urgency  function"  is 
defined  to  predict  the  points  at  which  the  pilot  switches  his 
attention  from  one  axis  to  another.  This  urgency  function 
consists  of  a linear  combination  of  the  state  variables  related 
to  each  axis.  It  is  assumed  that  the  pilot  devotes  his  atten- 
tion to  whichever  axis  has  the  highest  urgency  function  at  any 
particular  time. 

The  model  also  includes  a representation  of  crossfeed 
effects  between  the  control  inputs  on  the  various  axes.  The 
determination  of  these  crossfeed  effects  and  the  various 
crossover  model  gains,  leads  and  lags  is  accomplished  in  con- 
ventional ways,  but  determination  of  the  coefficients  of  the 
urgency  functions  requires  a complicated  parameter  search 
technique . 

For  the  primary  task  to  which  this  model  has  been  applied- 
a fourth-order,  two-axis  VTOL  hovering  task-stability  is  a major 
problem,  and  only  a small  set  of  urgency  function  parameters 
were  found  to  yield  stable  model  behavior.  The  switching 
behavior  implied  by  the  urgency  functions  was  found  to  match 
very  well  that  observed  in  experiments  with  a well-trained 
pilot . 


Input  Parameters 

The  controlled  process  dynamics  must  be  specified,  along 
with  experimentally  observed  control  crossfeed  effects  between 
axes . 
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Model  Outputs  . 

After  optimisation , the  model  yields  not  only  predicted 
mean  squared  tracking  errors  and  control  inputs,  but  also 
predicted  switching  statistics  between  axes. 


Validation 


For  the  VTOL  hover  task  noted  above,  the  model  successfully 
predicted  the  mean  values  of  all  state  variables.  It  also  pro- 
duced histograms  of  time  spent  controlling  each  axis  that  were 
very  similar  to  those  observed  with  a well-trained  pilot.  The 
outer-loop  state  variable  time  histories  produced  in  the  digital 
simulation  were  qualitatively  very  similar  to  those  observed 
in  the  experiments  with  the  pilot. 


Comments 


Onstott  observes  that  particularly  in  difficult,  multi-axis 
tracking  tasks,  it  is  essential  to  model  the  interactions 
between  the  state  of  the  system  and  the  pilot's  switching  of 
attention  from  one  axis  to  another.  In  the  specific  case 
studied,  the  system  dynamics  were  such  that  only  a narrow  range 
of  urgency  function  parameters  produced  stable  system  behavior. 
In  such  cases,  mathematical  optimisation  is  likely  to  yield 
models  that  match  very  well  with  human  performance.  In  other 
cases,  where  system  performance  is  relatively  insensitive  to 
changes  in  the  model  parameters,  it  will  not  be  possible  to 
identify  model  parameters  so  precisely;  under  these  conditions, 
however,  the  issue  may  be  academic. 
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Description 

This  model  is  based  upon  information  theoretic  concepts 
developed  by  Shannon  and  others.  Fundamentally,  it  assumes 
that  a human  operator's  fixation  frequency  for  a particular 
instrument  or  display  is  dependent  upon  its  information  genera- 
tion rate, 

- A2 

H = W log2  -~2  bits/sec  , 

E 

where  W is  the  cutoff  frequency  of  the  displayed  signal,  A is 
its  amplitude,  and  E is  the  permissible  rms  reading  error. 

For  an  observer  with  a fixed  channel  capacity,  who  must  share 
his  attention  among  several  displays  presenting  random,  un- 
correlated signals  with  known  information  rates,  the  atten- 
tional  demand  of  a particular  instrument  is  calculated  to  be 

A. 

T.  = 2K  W.  log-  — - + 2W . C sec/sec  , 
l l J2  E . l 

l 

where  Tf  is  the  proportion  of  total  time  that  must  be  devoted 
to  display  i,  K is  a constant  with  dimensions  of  time  per  bit, 
and  C (with  dimensions  of  time  per  fixation)  is  a constant  that 
accounts  for  movement  time  and  minimum  fixation  time. 

Using  this  latter  relationship,  the  total  workload  placed 
on  an  ideal  observer  by  a given  set  of  displays  can  be  computed. 
Decisions  can  be  made  regarding,  for  example,  the  effects  on 
observer  workload  of  adding  an  additional  display. 
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For  cases  in  which  the  information  transmitted  by  the  various 
displays  is  independent  and  uncorrelated,  and  in  which  a random 
scanning  pattern  is  assumed,  the  probability  of  a scanning 
transition  between  ciny  two  displays  is  shown  to  be  simply  the 
product  of  their  relative  fixation  frequencies.  In  general, 
however,  it  appears  that  scanning  patterns  are  not  random. 

In  an  aircraft  cockpit,  for  example,  a deviation  on  one  instru- 
ment will  be  closely  related  to  deviations  on  others,  and  the 
pilot  may  employ  several  conditional  scanning  patterns  that 
depend  upon  the  instrument  readings  encountered.  Even  in 
cases  where  the  displayed  information  is  completely  independent, 
scanning  patterns  may  be  modified  on  the  basis  of  recent 
readings:  the  observer  will  tend  to  return  his  attention  more 

frequently  to  display  showing  a near-critical  reading. 

To  address  the  latter  issue,  a model  was  proposed  in  which 
the  controller  computers  the  time  intervals  at  which  each  signal 
reaches  a certain  probability  of  exceeding  a threshold  limit, 
and  defers  additional  observations  of  these  signals  until  the 
respective  time  intervals  have  elapsed.  A possible  Markov  model 
of  controller  behavior  was  then  outlined,  by  which  improved 
estimates  of  fixation  frequencies  and  transition  probabilities 
could  be  achieved.  Basically,  this  model  assumes  that  the 
transition  probabilities  from  display  i to  display  j are  inde- 
pendent of  the  previous  sequence  of  transitions,  although  both 
the  transition  probabilities  and  the  dwell  times  may  be  functions 
of  both  i and  j. 


Input  Parameters 


To  estimate  the  scanning  statistics  of  the  human  controller, 
the  bandwidths  and  amplitudes  of  the  various  displayed  variables 
must  be  specified,  along  with  their  acceptable  limits.  Values 
for  the  constants  K and  C must  be  deduced  from  data  collected 
in  a specific  context. 


Model_  Output 

The  model  yields  estimates  of  the  fixation  frequencies  and 
durations  for  each  displayed  signal,  and  for  the  probabilities 
of  transitions  between  signals. 


Vnli da t ion 


Laboratory  experiments  were  conducted  using  a set  of  meters 
displaying  signals  of  various  amplitudes  and  bandwidths.  Subject 
were  required  to  signal  with  pushbuttons  whenever  a meter  reading 
exceeded  a certain  limit.  Fixation  frequencies  and  durations 
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were  obtained  and  compared  with  those  predicted  by  the  model. 

In  general,  the  agreement  was  good.  An  attempt  was  made  to  take 
into  account  statistical  correlations  between  displayed  signals, 
with  only  limited  success. 

Another  set  of  experiments  was  conducted  in  an  aircraft 
simulator,  with  pilots  flying  various  types  of  missions.  Eye 
movement  data  were  recording  along  with  time  histories  of 
the  displayed  readings  on  each  instrument.  Model  predictions 
based  upon  the  amplitude  and  bandwidth  of  the  various  signals 
proved  accurate  for  some  pilots  in  some  phases  of  flight,  but 
not  for  others. 


Comments 


It  is  apparent  that  this  model  yields  good  predictions  of 
scanning  behavior  under  certain  circumstances.  The  failure  of 
the  model  to  account  for  other  results  can  be  attributed  to 
(1)  its  failure  to  take  into  account  the  redundancy  of 
information  obtaineible  from  alternative  sets  of  instruments, 
which  permits  controllers  to  adopt  quite  different  alternative 
scanning  strategies  under  various  conditions,  and  (2)  its 
failure  to  take  into  account  the  interactions  between  control 
behavior  and  instrument  sampling,  which  permit  the  controller 
to  estimate  changes  in  the  displayed  signals.  A closely 
related  problem  is  that  particular  signals  become  more  or  less 
critical  during  different  maneuvers,  an  effect  thcit  is  largely 
ignored  by  this  model. 
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Smallwood  1 s Instrument  Monitoring  Model 
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D e scriptioi  i 

This  model  grew  out  of  previous  instrument  monitoring  studies 
involving  Senders,  Elkind,  Grignetti,  and  Smallwood.  The  task 
under  consideration  involves  a human  controller  monitoring  the 
readings  on  four  meters,  which  are  driven  by  signals  of  different 
amplitudes  and  bandwidths,  and  signalling  whenever  any  meter 
exceeds  a certain  threshold. 

The  model  assumes  that  the  monitor  constructs  an  internal 
model  of  the  processes  being  monitored,  and  that  at  any  instant 
he  can  estimate  the  probability  density  functions  of  the 
amplitude  of  each  signal.  This  p.d.f.  will,  of  course,  depend 
on  the  tine  elapsed  since  his  last  reading.  Immediately  after 
an  instrument  is  read,  its  p.d.f.  is  an  impulse;  i.e.,  the 
monitor  is  assumed  to  have  perfect  knowledge  of  its  amplitude. 

As  time  passes,  however,  the  variance  of  the  p.d.f.  increases 
and  its  mean  approaches  the  average  amplitude  of  the  signal. 

As  this  occurs,  the  probability  that  the  signal  has  exceeded 
the  threshold  tends  to  increase.  The  model  assumes  that  each 
scanning  transition  is  to  the  instrument  for  which  the  probability 
of  exceeding  the  threshold  is  greatest. 

The  model  further  assumes  that  a "dead  time"  of  0.1  second 
is  required  to  shift  attention  between  two  nstruments  and  that 
the  time  required  to  road  an  instrument  is  inversely  related  to 
its  distance  from  the  threshold.  The  precise  form  of  this 
relationship  must  be  deduced  from  data  collected  in  the 
appropriate  context  . 


Input  Parameters 

A statistical  description  of  each  displayed  signal  is 
required,  along  with  its  respective  threshold.  Overall  fixa- 
tion frequency  must  bo  matched  to  that  observed  in  the 
appropriate  context . 
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Model  Output 

The  model  yields  predictions  of  the  relative  fixation 
frequency  and  duration  of  fixation  for  each  instrument,  as 
well  as  predictions  of  the  average  transition  probabilities 
between  the  instruments. 


V a lid  a t .i  on 

The  model  predictions  were  compared  v/ith  data  obtained 
by  Senders  in  another  experiment  (see  Abstract  No.  12  ) . 
parameter  was  adjusted  to  match  the  observed  overall  scanning 
frequency,  the  model  output  matched  the  experimental  results 
fairly  well.  A second-order  version  of  the  human  controller's 
model  (i.e.,  one  that  took  signal  velocity  into  account  along 
with  signal  amplitude)  produced  a better  overall  fit  than  a 
first-order  version. 


Comments 


This  model  represents  an  improvement  over  Sender's  model 
in  that  the  effect  on  scanning  behavior  of  the  proximity  of 
a displayed  signal  to  its  critical  threshold  is  explicitly 
taken  into  account.  No  attempt  was  made  to  model  the  effects 
of  possible  cost  differences  in  threshold  passages  of  different 
signals.  The  interactions  between  the  displayed  signals  and 
the  operator's  control  inputs  were  also  excluded  from 
consideration . 
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C arbonoll’s  Queueing  Node!  _o f Visual  Samp ling 
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Peso ription 


This  nodel  for  a pilot's  instrument  scanning  behavior  is 
based  upon  queueing  theory  concepts.  The  observer  is  modeled 
as  a single-channel  server  that  can  attend  to  only  one  instrument 
at  a time.  It  is  assumed  that  at  each  step  in  his  scanning 
process,  the  observer  tries  to  minimize  the  risk  that  an  instru- 
ment to  which  he  is  not  attending  will  exceed  some  threshold 
reading.  It  is  further  assumed  that  the  time  involved  in  reading 
each  instrument  is  constant  (approximately  1/3  second) , and  that 
whenever  a longer  observation  time  is  noted,  it  is  because  the 
observer  has  chosen  to  look  at  the  same  instrument  again  to 
minimize  total  cost. 

The  total  cost  of  not  looking  at  any  instrument  is 
defined  to  be 


C(t) 


M 

E 

i=l 


C. 

i 

1 


Pi  (t) 

- Pi  ( t) 


where  M is  the  total  number  of  instruments,  C j is  the  cost 
associated  with  exceeding  the  established  threshold  for  instru- 
ment i,  and  Pi.(t)  is  the  probability  that  instrument  i will 
exceed  its  threshold  at  time  t.  The  total  cost  of  looking  at 
instrument  j at  time  t is 


C ' ( t ) = C(t)  - Cj  Pj  (t) 

The  model  assumes  that  the  pilot  chooses  the  instrument  j 
that  will  minimize  C * ( t)  at  any  instant.  Each  instrument  is 
treated  as  a member  of  a set.  competing  for  the  pilot's  atten- 
tion. The  nonlinear  dependence  of  cost  on  the  probability  of 
an  instrument's  exceeding  a threshold  insures  that  every 
’ n st.  rumen  t will  eventually  reach  a point  where  it  must  be 
. ' t nded  to. 
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A significant  feature  of  the  model  is  that  the  observed 
signals  are  not  assumed  to  be  cjaussim  with  zero  mean.  This 
permits  the  mode]  to  deal  with  signals  that  are  modified  by 
feedback  loops  involving  the  pilot's  control  actions. 

The  model  is  implemented  as  a computer  simulation,  which 
can  be  exercised  using  different  assumptions  regarding  cost 
thresholds,  signal  characteristics,  etc. 


Input  Paramet e rs 

To  exercise  the  model,  one  must  specify  the  statistical 
characteristics  of  each  displayed  signal,  the  costs  of  exceeding 
given  thresholds  on  each  display,  and  the  thresholds  below  which 
each  instrument  reading  is  ignored. 


Mo d el  Outpu t s 

The  model  yields  a time  sequence  of  instrument  fixations, 
which  may  be  abstracted  to  identify  patterns  of  scanning  activity. 


Val i!  at ion 


This  model  was  applied  to  eye  movement  data  obtained  from 
several  experienced  pilots  in  an  instrument  flight,  simulator. 
Using  estimates  obtained  from  the  pilots  of  the  relative  cost 
of  various  instrument  deviations  in  different  phases  of  flight, 
a computer  simulation  of  predicted  scanning  behavior  was  run. 

The  results  of  this  simulation  agreed  quite'  well  with  the 
experimental  data  - much  better  than  did  simpler  scanning  models 
based  only  on  instrument  bandwidths.  The  model  was  found  to  be 
particularly  valuable  as  signal  variances  and  task  difficulty 
increased,  conditions  in  which  greater  scanning  aperiodicity  is 
usually  observed. 


Common  !:s 

This  model  represents  a significant  advance  in  modelling 
instrument  scanning  behavior.  It  includes  not  only  differential 
costs  for  deviations  on  various  instruments,  but  also  a method 
for  incorporating  the  effects  of  the  pilot's  c trol  activities. 
The  model  does  not  attempt,  however,  to  take  into  consideration 
cross-coupling  among  instruments,  though  Carbonoll  asserted 
that  such  an  extension  is  possible.  The  flexibility  and  power 
of  this  model  is  obtained  at  the  cost  of  considerable  analytical 
complexity.  It  appears  that,  the  complexities  of  scanning 
behavior  can  adequately  bo  studied  only  through  extensive 
s i mu  1 a I i on . 
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opment of  a new  link  value  estimator.  Proceedings  of  the 
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June  1968. 

Description 


This  model  treats  the  processes  involved  in  scanning, 
selecting,  sampling  and  reconstruction  of  signals  displayed  on 
a multi-instrument  panel  by  adding  a "perceptual"  bloc):  to  the 
classical  quasi-linear  model  of  the  pilot.  This  added  block  is 
stipulated  to  account  for  the  effects  of  scanning  and  sampling, 
such  efforts  be  defined  as  the  ratio  between  behavior  under 
continuous,  full-foveal  tracking  and  the  actual  sampled  tracking, 
in  each  of  the  multiple  loops  of  control. 

The  model  is  based  on  the  following  fundamental  assumptions: 
1)  scanning  and  sampling  in  closed  loop,  multi-variable  control 
can  be  accounted  for  by  models  that  are  extensions  of  quasi-linear 
models  used  for  non-scanning  cases;  2)  scanning  behavior  is  stable 
in  a statistical  sense;  3)  sampling  of  a given  display  is 
assumed  to  be  almost  periodic  with  appreciable  statistical 
flucLuations  that  randomize  the  data;  and  4)  high  frequency 
effects  of  scanning  sampling  and  reconstruction  give  rise  to  a 
broadband  "sampling"  remnant  that  can  be  modelled  as  an  injected 
noise  at  the  pilot's  input  (i.e.,  "observation  noise"). 

The  resulting  model  is  comprised  of  a quasi-linear  random 
input,  "perceptual  describing  function"  which  multiplies  the 
basic  describing  function  and  a broadband  sampling  remnant  which 
adds  to  the  basic  remnant.  The  perceptual  describing  function 
consists  of  an  attenuation  factor  and  an  equivalent  sampling  and 
reconstruction  delay.  The  spectrum  of  the  injected  sampling  .is 
proportion.il  to  the  mean-squared  value  of  the  signal  being  sampled 
and,  for  frequencies  well  below  the  average  sampling  frequency, 
is  approximated  by: 


where  x .is  the  observed  signal,  T the  average  sampling  interval, 
Tq  the  dwell  time  and  T0  the  minimum  sampling  interval. 


Input  Parameters 

Application  of  the  model  requires  all  information  necessary 
for  a manual  control  analyis,  i.e.,  a complete  system  description 
and  a model  for  manual  control.  The  system  description  must 
include  mission  description  and  criteria,  vehicle  dynamics, 
disturbances  and  commands  in  a suitable  form.  The  manual  control 
model  is  the  multi-loop  quasi-linear  describing  function  model 
modified  to  include  the  "perceptual"  describing  function. 


Model  Output s 

The  outputs  f the  model  are  those  of  a standard  manual 
control  analysis  plus  quantities  related  to  display  design  and 
utilization  such  as  fixation  frequencies  and  durations,  scanning 
patterns,  scanning  workload  metrics  and  preferred  presentation 
and  arrangement. 

Model  Validation 

The  model  has  been  used  to  analyze  a B707-320  in  ILS  approach 
and  predicted  display  arrangements  agree  quite  well  with  those 
adopted  for  an  American  Airlines  aircraft  (McRuer,  et.  al)  . How- 
ever, detailed  validation  for  such  a complex  task  is  not  available. 
The  model  has  cilso  been  tested  in  carefully  structured  two 
display  cases  (Allen,  Clement  and  Jex)  with  encouraging  results. 


Comment s 

The  strengths  of  the  model  are  those  of  the  elemental  models. 
However,  the  drawbacks  of  the  individual  models  may  be  compounded 
by  the  syntheses. 
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Optimal  Control  Scanning  Model 
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Description 

The  optimal  control  scanning  model  is  a method  for  including 
scanning  constraints  in  the  optimal  control  framework.  The  model 
is  closely  related  to  the  concepts  in  Levison's  attention-sharing 
model.  The  main  difference  is  that  the  scanning  model  accounts 
for  perceptual  losses  associated  with  large  eye  movements. 

The  basic  idea  is  to  assume  that  the  observation  noise 
process  depends  parametrically  on  the  fixation  point  of  the 
eye,  represented  mathematically  by  the  parameter  w.  A scanning 
strategy  is  defined  as  a method  for  picking  this  fixation  point 
at  different  time  instances,  i.e.,  a procedure  for  choosing  m(t). 
It  is  assumed  that  the  pilot's  fixation  point  cannot  be  changed 
instantaneously  so  that  there  is  a finite  transition  time  tQ , 
associated  with  each  such  change.  The  transition  time  also 
includes  time  lost  due  to  the  apparent  loss  of  perception  that 
occurs  immediately  before  and  after  each  eye  movement.  Note 
that  co(t),  so  defined,  is  piecewise  constant  and,  over  any  time 
interval,  may  be  described  by  a scan  sequence  of  the  form  (say) 
{1,0, 2,0,1, } . 

When  the  fixation  point  of  the  eye  varies  with  time,  the 
observation  noise  covariance  matrix  V(l,  will  also  vary  with  time. 
However,  for  each  value  of  w there  is  a corresponding  constant 
observation  noise  covariance  matrix.  If  Vj  is  the  covariance 
matrix  corresponding  to  m--i  , then  the  diagonal  element  in  the 
j th  row,  Vi ( ) ) , is  the  noise-covariance  associated  with  viewing 
the  j tli  displayed  variable  while  fixating  fovenlly  display  i. 

Note  that  each  of  the  matrices  Vj  have  elements  corresponding 
to  viewii.  j each  of  the  displayed  variables.  Some  of  these 
elements  will  correspond  to  foveal  viewing  and  others  to 
peripheral,  viewing.  Put  another  way,  all  the  displayed 
variables  are  assumed  to  be  observed  continuously,  but,  in 
general,  the'  signals  on  Ih  > "fixated"  display  will  have  low  a 
noise  levels  associated  with  t he i i observation;  this  influences 
the  scanning  sti stogy.  In  formulating  the  method  for  predicting 
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scanning  behavior,  it  is  assumed  that  the  Matrices  V^,  j=/o,  are 
given.  The  elements  of  the  matrix  VQ  are;  assumed  to  be  infinite, 
i.e. , that  nothing  is  seen  while  the  operator  is  switching  points 
ot  f i xa  t ion  . 

It  is  also  assumed,  without  loss  of  generality/  that  the 
scanning  strategy  is  periodic  with  a fixed,  but  arbitrary, 
scan  period  T.  It  is  then  possible  to  show'  that  for  a given 
scanning  strategy,  o , the  quadratic  performance  criterion  of  the 
optimal  control  model  depends  implicitly  on  w . Therefore, 
consistent  with  the  optimality  hypothesis,  it  is  assumed  that 
the  pilot  selects  his  control  (u)  and  scanning  strategy  (m)  to 
jointly  minimize  the  performance  criterion. 


flodel  Inputs 

In  addition  to  those  inputs  necessary  for  the  optimal  control 
model,  one  must  provide  estimates  of  the  average  transition  times 
betw’een  instruments  and  estimates  for  observation  noise  covariances 
for  peripheral  viewing  of  displays.  (The  simplest  assumption  is 
that  nothing  is  seen  in  the  periphery.) 


Model  Out put s 

In  addition  to  the  standard  outputs  available  from  the 
optimal  control  model,  one  obtains  estimates  of  average  scanning 
behavior.  The  quantity  tp  can  be  identified  with  mean  observation 
times  for  display  i (for  all  i).  The  "predicted"  average  scan 
period  is  the  optimal  value  of  T.  It  will  be  noted  that  errors 
in  the  prediction  of  scan  period  will  inevitably  result  in  errors 
in  "predicting"  mean  observation  times.  Therefore,  it  is  better 
to  consider  the  fraction  of  time  spent  fixating  display  i 
foveallv  (ff  = ti/T)  instead  of  the  mean  observation  time  on 
that  display  (t£) . The  fractions  of  fixation  and  the  average 
scan  period  are  quantities  that  can  be  compared  directly  with 
values  obtained  from  measured  eye-movement  data.  If  it  is 
assumed  that  instrument  transitions  form  a Markov  chain  in  which 
the  observer  draws  at  random  from  the  set  of  displayed  signals 
with  probabilities  equal  to  the  fixation  fractions  fj  each  time 
a display  transition  is  to  bo  made,  then  transition  link  pro- 
babilities and  average  dwell  times  can  be  computed.  (The 
fixation  fractions  are  then  identified  with  fixation  probabi- 
lities, which  is  tantamount  to  assuming  ergodicity  of  the 
scanning  process.) 


Vn 1 idn t ion 

Validation  results  for  the  optimal  scanning  models  are  very 
meager,  b i ng  limited  to  a demonstration  that  operator  performance 
in  <i  VTOb  hover  Iasi,  with  two  sopjrutod  displays  could  bo  reproduced. 
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Co mmen  Ls 

The  advantages  of  thi.s  approach  to  scanning  prediction  are 
several.  The  sampling  model  is  such  that  the  human's  monitoring 
behavior  depends  upon  the  control  requirements  and  control 
actions  in  an  explicit  way.  Control  and  scanning  strategies 
are  determined  at  essentially  the  same  time;  one  avoids  the 
inherently  iterative  and  judgmental  procedures  involved  in  a 
process  thuit  requires  the  specific  loop  topology  to  be  known 
before  computing  scanning  parameters.  Also,  one  avoids  separate 
assumptions  concerning  reconstruction  of  the  sampled  signal; 
the  Kalman  estimator  performs  this  function,  cis  in  the  no-scanning 
case.  Other  advantages  include  the  straightforward  manner  in 
which  peripheral  vision  may  be  considered  and  the  lack  of  any 
restrictive  assumptions  concerning  coupling  of  signals  on  various 
display  indicators. 

The  principal  difficulty  associated  with  the  approach  would 
appear  to  be  the  computational  burden  in  solving  the  optimal 
sampling  problem;  it  should  be  noted,  however , that  solving  the 
optimization  problem  may  not  bo  more  expensive  than  the  computa- 
tional procedures  associated  with  other  methods  for  predicting 
scanning  in  closed  loop  control  tasks.  In  the  two-display  case 
numerical  search  techniques  are  adequate  and  not  too  expensive. 

For  more  than  two  displays,  search  methods  arc  not  too  practical 
and  a suitable  optimization  algorithm  is  required. 
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Cont rol  Theory  Model  for  Deci sion  Making 
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Description 


This  is  an  extension  of  the  optimal  control  model  of  the 
human  operator  to  tasks  involving  continuous  detection  and/or 
decision-making.  The  model  is  identical  to  the  optimal  control 
model  in  terms  of  its  assumptions  concerning  information 
processing.  Thus,  it  is  assumed  that,  because  of  inherent 
limitations,  the  operator  perceives  a noisy,  delayed  version 
of  the  displayed  variables.  The  perceived  data  are  then 
"processed",  via  an  optimal  estimator/predictor  combination, 
to  generate  a minimum  variance  estimate,  x(t) , of  the  system 
state  vector  and  the  covariance  of  the  error  in  that  estimate, 

Z_(t)  . The  pair  (x(t),  E_(t)  is  a sufficient  statistic  for 

testing  hypotheses  about  the  state  of  the  system. 

The  model  assumes  that  the  operator  is  an  optimal  decision- 
maker in  the  sense  of  maximizing  expected  utility.  This  strategy 
is  then  applied  to  the  problem  of  deciding  whether  or  not  a signal, 
corrupted  by  noise,  is  within  certain  prescribed  tolerances.  For 
equal  penalties  on  missed  detections  and  false  alarms,  this  rule 
reduces  to  one  of  minimizing  the  expected  decision  error.  The 
resulting  decision  rule  is,  simply,  a likelihood  ratio  test. 


Model  Inputs 


Model  inputs  include  those  necessary  to  apply  the  optimal 
control  model  with  the  exception  of  motor-noise  and  the  quadratic 
cost  functional.  Instead  of  the  latter,  one  must  specify  the 
decision  cost,  i.e.,  the  utilities  for  various  correct  and 
incorrect  decisions. 


Rcpoi t No.  3446 


Bolt  Beranek  and  Newman  Inc. 


Mo  del  Ou  typu  ts 

Many  measures  of  decision  or  detection  performance  can  be 
predicted.  Thus,  one  can  compute  the  decision  error  probabilities 
or  other  measures  such  as  the  distribution  function  for  failure 
detection  time.  It  should  be  emphasized  that  the  model  can  be 
used  to  explore  the  effects  on  decision  performance  of  changes 
in  system  or  display  parameters. 


Validation 

Experimental  results  have  been  compared  with  model  predictions 
for  the  following  task  situations  (1)  single  decision  task;  (2) 
two  decision  tasks  and  (3)  concurrent  manual  control  and  decision 
tasks.  Using  fixed  values  for  model  parameters,  we  can  predict 
single-task  and  two-task  decision  performance  scores  to  within 
an  accuracy  of  10  percent.  Interference  between  concurrent 
decision  and  control  tasks  is  revealed,  but  the  results  of  this 
experiment  did  not  allow  a conclusive  test  of  the  predictive 
capability  of  the  model. 


Comments 

One  can  consider  decision  making  as  being  comprised  of  two 
processes,  the  generation  of  a decision  function  (i.e.,  the 
process  of  converting  actual  observations  into  one  or  several 
numbers)  and  the  formulation  of  a rule  or  lav;  which  uses  the 
output  of  the  decision  function  to  decide  between  hypotheses. 

In  this  model,  the  information  processing  structure  of  the  optimal 
control  model  generates  the  decision  function  and  the  decision 
rule  is  essentially  that  of  a Baysian  decision  maker.  It  is 
clear  that  other  decision  rules  may  be  formulated  (prescriptive 
or  descriptive).  These  will  lead  to  somewhat  different  models 
but  all  will  incorporate  an  explicit  information  processing 
model  that  is  based  on  the  optimal  control  model. 

The  decision  making  model  has  only  received  limited  valida- 
tion. Moreover,  there  were  several  methodological  problems  in 
the  experiments  used  for  validation.  These  are  discussed  in  the 
cited  reference.  Thus,  the  model  needs  further  verification  before 
it  can  be  used  with  confidence. 
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Description  of  Model 

This  model  addresses  the  problem  of  human  adaptation  to 
sudden  changes  in  controlled  process  dynamics  in  a compensatory 
tracking  task.  It  attempts  to  predict  when  a human  controller 
will  first  become  aware  of  an  unanticipated  change  in  dynamics 
(such  as  the  failure  of  a stability  augmentation  system),  and 
how  he  will  change  his  control  behavior  to  compensate  for  the 
transition . 

The  model  assumes  that  the  controller  is  well-trained  and 
familiar  with  all  of  the  possible  changes  in  the  dynamics  that 
can  occur.  It  postulates  that  he  continuously  monitors  the 
change  in  the  system  error  rate,  Ae,  that  results  from  his 
control  inputs,  and  compares  this  change  with  the  predicted 
change  that  he  expects  on  the  basis  of  his  experience  with  the 
system  dynamics.  If  the  dif  ference  between  the  observed  and 
expected  changes  in  error  rates  exceeds  a criterion  value,  the 
model  reports  that  a change  in  system  dynamics  has  occurred. 

The  model  then  enters  a Bayesian  analysis  phase  in  which 
the  controller  compares  the  observed  change  in  error  rate'  with 
th.it  expected  under  alternative  hypotheses  concerning  the  system 
change  that  has  occurred  and  selects  that  sot  of  alternative 
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dynamics  that  best  accounts  for  the  observed  results.  Usually, 
only  one  candidate  will  match  adequately,  but  if  not,  identifi- 
cation is  postponed  and  additional  observations  are  accumulated 
until  an  unequivocal  choice  can  be  made. 

The  model  asserts  that  the  controller  then  carries  out.  a 
rapid,  often  pre-programmed  series  of  control  movements  to  null 
accumulated  errors,  and  finally  conducts  a "fine-tuning"  process 
to  optimize  his  transfer  function  for  the  new  controlled  process 
! dynamics. 


Input  Parameters 

The  model  assumes  precise  knowledge  of  the  transfer  function 
of  the  "normal"  controlled  process  and  each  of  the  alternative 
"failed"  processes.  It  also  requires  statistical  descriptions 
of  the  disturbance  signal  being  tracked  and  of  the  resulting 
error  signal.  The  steady-state  tracking  characteristics  of  the 
human  controller  are  assumed  to  be  consistent  with  the  "crossover 
model"  (see  Abstract  No.  dT  ) . 


Model  Outputs 

For  well-defined  tracking  tasks,  the  model  predicts  the 
time  required  to  detect  and  respond  to  various  changes  in  con- 
trolled dynamics.  With  sufficient  knowledge  of  specific  tracking 
situations,  the  probability  of  an  erroneous  response  can  also  be 
predicted . 


Model  Validation 


The  model  has  been  applied  to  data  obtained  in  compensatory 
tracking  tasks  conducted  in  the  laboratory  using  K,  K/s,  and 
K/s"  controlled  dynamics.  Transitions  studied  included  gain 
increases,  gain  decreases,  and  gain  reversals,  as  well  as 
changes  in  the  order  of  the  controlled  dynamics  (e.g.,  K/s^  to 
K/s).  Experiments  were  conducted  with  up  to  18  possible  tran- 
sitions, and  also  with  transient  disturbances  designed  to  mimic 
the  error  waveforms  of  an  actual  transition. 
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The  model  successfully  predicted  the  response  times  of  the 
various  transitions  and,  in  many  cases,  accounted  for  errors  in 
identifying  the  transition  that  had  occurred. 


Comments 

The  utility  of  this  model  lies  primarily  in  its  capability 
to  account  for  the  detailed  behavior  of  well-trained  human 
subjects  responding  to  sudden  changes  in  controlled  dynamics. 

It  represents  a "best  case"  model  of  this  behavior;  in  most 
realistic  tracking  situations,  however,  the  controller  is  not 
aware  of  all  possible  malfunctions  and  is  not  well-practiced  in 
responding  to  them.  Therefore,  this  model  is  of  limited  use- 
fulness in  predicting  the  consequences  of  system  failures. 
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De scr i p t ion 

Levi  son's  model  for  attention-sharing  is  intended  to  apply 
to  situations  (such  as  continuous  manual  control)  in  which  the 
human  is  required  to  operate  as  a continuous  processor  of 
information.  The  model  is  further  restricted  to  situations  in 
which  task  interference  can  be  assumed  to  be  of  central  origin. 
(Visual  scanning  effects  can  be  treated  in  an  analogous  fashion.) 
This  model  serves  as  a theoretical  framework  for  including 
disruptive  effects  of  workload  on  operator  performance. 

Levison' s model  follows  directly  from  his  model  for  controller 
remnant  or  randomness.  That  model  states  that  each  variable 
"perceived"  by  the  operator  is  perturbed  by  a white,  gaussian 
noise  process  with  autocovariance  given  by 

V.(t)  = IT  P.  0?(t)  (1) 

where  Uj(t)  is  the  variance  of  the  i^1  perceived  variable  and  Pj  is 
the  "noise/sicmal  ratio"  and  has  units  of  normalized  power  per 
rad/sec.  Numerical  values  for  Pj  of  0.01  (i.e.,  -20  dB)  have 
been  found  to  bo  typical  of  a variety  single-variable  control 
situations.  The  relative  invariance  of  Pj  with  control  task 
parameters  suggests  that  the  basic  observation  noise  defined  by 
Equation  (1)  represents  a processing  limitation  of  the  human 
operator. 

The  model  for  attention  sharing  (or  task  interference) 
assumes  that  if  two  or  more  display  variables  are  to  be 
processed,  the  noise/signal  ratios  may  deviate  from  their 
nominal  values.  (No  deviation  is  assumed  when  position 
and  rate  information  are  obt  lined  from  the  same  display  or 
w h ■ ■ n the  display  .is  integrated.)  Specifically,  t he  model  for 
interference  assures  that  is  the  human  devotes  less  "attention" 
to  i giver,  displayed  variable  th  ’ no  i s a on  that  variable 
increases  proportion.il  1 y . Thu.,  it  Pt,  is  tic  noise- ratio 
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when  "full  attention"  is  devoted  to  a task,  then  when  the  subject 
is  forced  to  pay  less  than  full  attention  to  the  ith  variable, 
tiie  effective  noise/signal  ratio  is 
P 

„ __  o 


where  f^  is  the  fr£ictio.n  of  attention,  and  0 <_  fj_  < 1,  f ^ = 1 . 
Clearly,  because  of  tiie  increase  in  noise,  attention  sharing  will 
degrade  performance  reliability. 

Equation  (2)  can  be  used  to  help  predict  the  effects  of 
attention-sharing  on  performance.  To  predict  what  happens  on  a 
specific  task  when  only  partial  attention  can  be  paid  to  it, 

(2)  in  conjunction  with  (1)  is  used  to  determine  the  appropriate  ob- 
servation noise  covariance.  Then  the  optimal  control  model  is  used 
to  predict  pilot  behavior  and  overall  system  performance.  The 
optimum  allocation  of  attention  is  determined  and  provides  the 
basis  for  prediction  of  the  other  measures  of  interest. 

The  model  of  task  interference  lends  itself  straightforwardly 
to  the  prediction  of  the  amount  of  "workload"  associated  with  a 
given  task.  The  "workload  index"  (WI)  is  defined  as  the  fraction 
of  the  controller's  capacity  that  is  required  to  perform  a given 
task  to  some  specified,  or  criterion,  level  of  performance. 

This  metric  can  be  predicted  quantitatively  with  the  existing 
implementation  of  the  optimal-control  model.  The  procedure  is 
identical  to  that  for  predicting  task  interference;  once  the 
model  is  "calibrated"  for  single-axis  behavior  (either  by  doing 
a simple  experiment  or  by  using  nominal  values  of  parameters  that 
match  previous  data),  a curve  of  performance  sc  - • versus  observa- 

tion noise  ratio  is  obtained.  By  relating  the  observation  noise 
ratio  to  fraction  of  capacity,  a quantitative  value  of  workload 
may  be  determined. 


Mode 1 Inputs 

Inputs  to  the  model  are  the  base  noise/signal.,  P0 , and  t lie 
definition  of  those  variables  (or  tasks)  that  are  assumed  to  be 
mutually  interfering.  Of  course,  all  inputs  necessary  for 
applying  the  optimal  control  model  must  also  be  specified. 


Mo  tic  • 1 Out  put  s 

In  addition  to  those  system  performance  measures  provided  by 
the  optimal  control  model,  the  model  for  attention-sharing  yields 
the  optimal  allocation  of  attention  to  the  various  tasks.  If 
performance  spcci f ications  are  available,  a prediction  of  workload 
is  also  made. 
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Va  1 idat  i on 

The  model  for  task  interference  has  been  validated  in  basic 
two-axis  and  four-axis  tracking  tasks.  It  has  also  been  validated, 
but  to  ci  lesser  extent,  in  a two-variable  decision-making  task. 

The  effects  of  interference  on  total  performance  appear  to  be 
predicted  with  excellent  accuracy.  However,  sub-task  scores  do 
not  agree  so  well  which  suggests  that  the  subjects'  allocation 
of  attention  may  deviate  from  that  predicted  by  the  model. 


Comments 


There  are  several  basic  problems  associated  with  application 
of  the  task  interference/workload  models.  First,  there  is  the 
necessity  for  specifying  a "reference"  noise/signal  ratio  PQ. 
Ideally,  PQ  should  correspond  to  full  attention.  However, 
conducting  an  experiment  to  measure  PQ  does  not  seem  possible. 

Thus,  one  uses  the  value  of  Po  obtained  in  standardized 
laboratory  tracking  tasks  in  which  the  subjects  are  well-trained 
and  highly  motivated  to  minimize  tracking  error.  It  is  assumed 
that  this  value  of  PQ  corresponds  to  a "demand"  that  would 
correspond  to  a high  workload  outside  the  laboratory.  If  one 
is  interested  in  the  relative  change  in  workload  from  one  situation 
to  another,  as  is  often  the  case,  then  the  value  of  PQ  is  not  too 


A second  problem  involves  solving  for  the  optimum  allocation 
of  attention.  This  is  a non-trivial  computation  problem.  Inasmuch 
as  it  is  often  the  case  that  reasonably  large  deviations  from 
optimal  allocation  fractions  do  not  result  in  seriously  degraded 
performance,  it  is  frequently  desirable  to  choose  f|s  to  reflect 
some  hypothesized  allocation  of  attention.  This  leads  to  a 
descriptive  model  of  attention-sharing  rather  than  a prescriptive 
one.  The  fact  that  overall  performance  does  not  suffer  much  when 
the  attention-split  deviates  from  optimal  is  one  reason  why  sub- 
task performance  is  not  so  well  predicted  by  the  model. 

A third  difficulty  arises  in  deciding  whether  variables  or 
tasks  do,  in  fact,  interfere.  This  is  especially  so  when  display 
information  is  presented  in  an  integrated  format,  e.g.,  in  a 
pictorial  display.  At  present,  sufficient  data  do  not  exist  to 
allow  one  to  determine  theoretically  the  degree  of  integration  of 
any  given  display.  However,  by  obtaining  model  predictions  for 
the  alternative  assumptions  of  complete  interference  and  no 
interference,  the  range  of  performance  to  be  expected  can  be 
predicted . 

Finally,  it  should  be  mentioned  that  the  model  for  attention- 
sharing did  not  a;  pear  to  hold  when  attempting  to  predict  inter- 
ference bet  we  -.ti  a tracking  task  and  a d -c  Lsion-mak  i ng  task  . 
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Gai's  Failure  Detection  Model 


Re f ore nee s 

Gai  , E.  G.  and  Curry,  R.  E. , "Failure  Detection  by  Pilots  During 
Automatic  Landing;  Models  and  Experiments,"  Proc.  of  Eleventh 
Annual  Conference  on  Manual  Control,  NASA  TMX- 6 2 , 464~,  Ames 
Research  Center,  Moffett  Field,  California,  May  1975. 

Gai,  E.  G. , "Psychophysical  Models  for  Signal  Detection  with  Time 
Varying  Uncertainty,"  Ph.D.  Thesis,  Department  of  Aeronautics 
and  Astronautics,  Massachusetts  Institute  of  Technology, 
January  1975. 


Description 

This  is  proposed  as  a model  of  the  pilot  as  a monitor  of 
instrument  failures  during  automatic  landing.  The  model  incorporates 
observation  noise,  a Kalman  estimator  and  a decision  rule  based  on 
sequential  analysis.  The  observation  noise  is  selected  according  to 
Lovison's  task  interference  model. 

The  Kalman  estimator  is  used  to  generate  the  decision  function. 

In  particular,  the  estimator  generates  estimates  for  the  state, 
x(t),  the  measurements,  y(t),  and  a measurement  residual 

e (t)  = y (t)  - y (t) 

where  y(t)  is  the  "noisy"  measurement.  In  addition,  covariances 
for  these  quantities  are  also  available  from  the  filter.  Under 
normal  operating  conditions  the  residual  is  a zero  mean  white 
gaussian  noise  process.  It  is  assumed  that  in  a failure# a 
deterministic  is  added  to  the  measurement.  In  that  case  the 
residual  is  still  white  and  gaussian  with  the  same  variance  as 
in  the  unfailed  case,  but  it  has  a non-zero  mean.  Thus,  Gai 
chooses  as  his  decision  function 

m 

A (m)  = i {?  (t . ) - 0. 
i=l  1 

where  > 0 is  the  mean  value  associated  with  a failure. 


The  decision  rule  that  is  used  is  a modified  form  of  the 
classical  sequential  analysis  rule,  namely 
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if  > (m)  - --  ---  choose  "failure" 


.ir  0 <_  A(m)  < — -j — take  another  observation 

if  X (m)  < 0 "reset"  X (m)  to  zero 


The  value  of  Ay  is  related  to  the  desired  false  alarm  and  miss 
probabilities  by 

A!  “ 1”  Ax  " 1 = - Un  A + (A-l)  ln  B/(l-B)] 


where 

A = (1  - P ( MS ) ) / P ( F A ) ; B = P (MS)  / (1-P  (FA)  ) 


Gai  claims  that  classical  sequential  analysis  cannot  be  applied 
to  the  failure  detection  problem  because  the  basic  assumption 
of  that  theory  is  that  the  same  mode  exists  during  the  entire 
period  of  testing. 


Mode I Inputs 

System  and  disturbances  must  be  given  as  in  the  optimal  control 
model  . In  addi  t ion,  observation  noise/signal.  ratio  for  the  human 
operator,  the  parameter,  0 , designating  the  mean  of  the  failed 
process,  and  the  "desired"  decision  performance  oarameters,  P(MS) 
and  P(I’A),  must  be  specified. 


Model  Outputs 


The  model  predicts  the  mean  and 
detection  tire  for  failure:..  These 
of  system  parameters. 


standard  deviation  of  the 

can  be  determined  as  a function 


I 
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Validation 


An  experiment  to  test  the  validity  of  the  model  was  conducted. 
In  this  experiment  subjects  had  to  detect  failures  in  the  glide- 
slope  and  airspeed  indicators  during  a simulated  landing  in  a 
Boeing  707  fixed  base  simulator.  The  results  showed  that  good 
agreement  between  predicted  detection  tines  and  experimental  data. 


Comments 

The  information  processing  part  of  the  optimal  control  model 
is  incorporated  in  this  model  except  that  the  time-delay  (and, 
therefore,  the  optimal  predictor)  are  ignored.  The  model  differs 
from  Levison's  model  for  decision-making  in  the  choice  of  filter 
output  for  a decision  function  and  in  the  decision  rule. 

It  should  be  noted  that  the  model  treats  only  instrument 
failures  that  are  evidenced  by  changes  in  the  mean  value  of  the 
display  variable. 
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Hunan  Operator  Simulator 


Ref e fences 

Strict,  M.  I.  The  Human  Operator  Simulator:  Volume  I 

Introduction  and  Overview  Analytics  Technical  Report  1117-1 
August  1975. 


Description 

The  goal  of  IIOS  is  to  provide  a computer  simulation  of  human 
performance  in  a goal-oriented,  task-processing  environment.  It 
is  used  in  conjunction  with  other  programs  such  as  the  Human 
Operator  Procedures  (HOPROC),  Human  Operator  Data  Analyzer/Collator 
(HODAC),  and  the  HOPROC  Assembler/Loader  to  provide  the  capabil ity 
for  full  simulation. 

Four  High  Level  Functions  are  defined: 

1.  The  decoder,  which  provides  the  translation  from  task 
requirements  to  human  performance  functions. 

2.  The  multiplexor,  which  sets  priorities  for  the  order  in 
which  task  requirements  wil 1 be  sequenced,  based  on  an 
index  of  criticality. 

3.  The  estimator,  which  is  the  sensing-remembering  core  of 
HOS  and  assigns  tine  costs  to  the  elemental  human 
performance  operations  undertaken.  It  includes  a 
memory  module  that  is  based  on  a strength  model  of 
memory  performance  having  a short-term  and  long-term 
memory  capability.  The  memory  module  is  the  major 
probabilistic  element  in  HOS.  The  estimator  also 
includes  a model  of  the  operator's  physical  and 
biomechanical  operations  such  as  eye  movements,  limb 
movenen t s , etc. 

4.  The  banker,  which  accumulates  time  costs  and  serves  to 

interface  HOS  with  the  simulated  hardware  components  of 
the  system.  , 

In  order  to  build  a man-machine  simulation  in  HOS,  the 
designer  starts  from  a task  analysis  and  writes  in  HOPROC  a 
procedural  description  of  the  activities  to  1)-  performed. 

Where  the  translation  from  procedures  to  human  performance 
functions  has  been  previously  defines:  for  generic  activities 
like  instrument  read  ing , only  parameter  valuer,  must  be  specif  i si. 


:>  o h 


However,  i * now  procedure:;  an-  on  >unterod  it  i.;  necessary  Lo 
provide  the  translation  i roin  prow  sure  to  he::;;:,  performance 
fund  i on..;  and,  where  nec;  ;s  iry,  n<  w hui  ti  j rfc  rmance  function 
def  ini  t i one. . 

In  terms  of  the  vocabulary  used  in  this  report  HOS  repre- 
sents n bottom-up  modelling  approach  that  carries  the  detail 
down  to  the  level  of  specifying  the  elemental  human  performance 
functions  out  of  which  more  molar  task-level  behavior  is  built. 


Input  Parameters 

In  the  extreme  case  the  model  user  mast  specify  the 
operational  procedures,  the  definition  of  th:  relationships 
between  tasks  and  human  performance  function.',  the  description 
of  the  human  performance  functions  and  the  parameter  settings 
that,  instantiate  the  models  at  each  of  thesv  levels  for  the 
actual  task  being  represented.  In  practice  the  user  may  take 
advantage  of  the  accumulated  catalo  , of  repr s-  ntations  at  these 
different  levels  and  specify  only  the  specific  parameters 
relating  to  his  simulation.  However,  where  new  representations 
are  required,  descriptions  of  the  models  the:. selves  must  also 
be  provided. 


Mod e 1.  Outp ut 

The  HODAC  program  makes  poss ible  user  sr  . ci f i cat  ion  of  any 
desired  output  variables  and  statist  ical  sum.  , mi'-s  of  them 
within  the  limits  of  the  level  of  detail  spec i *'ii  d in  the  models 
themselves . 


Model  Validation 

HOS  takes  a cumulative  increnv  vital  approach  to  model 
development  and  validation.  Validation  of  H'\"  may  be  undertaken 
in  successively  larger  units.  We  are  aw  ire  of  validation  to 
predict  m.  tor  reading  and  scanning  performance,  for  example. 

For  this  purpose,  the  models  required  for  th  :s  subset  of  tasks 
have  been  implemented  and  tested  against  r€‘ui  data.  When  those 
m >dels  have  been  tuned,  the  underlyin  j human  : ;i  >rn  mce 
components  are  available  to  predict  larger  units  of  behavior  of 
which  instrument  reading  is  a sub  , and  so  forth.  Wo  believe 
that  this  is  an  admirably  systomu*  c way  to  ; rocoed , but,  thus 
tar,  the  conduct  of  validation  st  . : -.  s c th  kind  lias  been 
I i mi  ted . 
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Comments 


The  cumulative  incremental  approach  to  development  and 
validation  that  the  underlying  structure  of  HOS  makes  possible 
is  an  important  feature  of  the  methodology.  A second  important 
contribution  is  the  insightful  and  careful  way  in  which  the 
various  components  of  human  performance  are  being  modu lar ized 
iind  interrelated.  As  a result,  many  of  the  aspects  of  human 
performance  fall  out  as  a consequence  of  these  logical 
reliitionships,  thereby  reducing  the  complexity  of  task 
modelling  requirements. 

The  ultimate  success  of  I10S  will  depend  on  two  factors: 

(1)  the  generality  with  which  individual  human  information 
processing  operations  can  be  modelled;  and  (2)  the  ability  to 
provide  an  integrative  structure  for  combining  elements  of 
human  performance  at  one  level  and  tasks  at  another  level  so 
that  the  model  is  predictive  of  system-level  performance. 
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Description 


Although  not  strictly  a model,  the  concepts  of  information 
measurement  may  be  used  to  formulate  models  for  a variety  of  human 
information  processing  activities.  The  fundamental  finding  is  that 
when  there  is  a one-to-one  mapping  of  stimuli  to  responses,  choice 
reaction  time  is  a linear  function  of  the  average  information 
(uncertainty  reduction)  when  responding  to  a stimulus  drawn  from  a 
we  1 1 -def ined  stimulus  set  whose  probabilities  of  occurrence  are 
known.  In  the  discrete  case,  the  average  information  is  defined 


as 


H 


ave 


n 

- X p.  log.. 
i-1  1 2 


(1  ) 


Ave . 


and 


RT 


a + b H 


(?) 
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This  finding  holds  for  varying  information  by  either  varying  the 
number  of  stimuli  in  the  set,  or  by  varying  the  relative  frequencies  of 
occurrence  of  particular  stimuli  in  the  set  and  then  varying  the 
first-order  sequential  probabilities  that  a particular  stimulus 
will  follow  any  other  stimulus. 

This  representation  works  best  when  the  stimuli  are  separate 
lights,  numbers,  or  other  unitary  events  that  either  occur  or  do 
not  occur.  However,  the  method  of  modelling  may  also  be  applied 
to  predict  the  time  to  read  a visual  display.  In  this  case,  one 
must  quantify  the  probability  distribution  of  alternative  display 
readings  using  either  the  discrete  or  continuous  version  of  the 
information  metric.  The  discrete  version  is  given  above.  The 
continuous  version  is  defined  by 


AVE 


= w log2 


A2 


where 


m = bandwidth  of  time  variation  in  meter  reading. 
A = rms  meter  reading  amplitude 
E = rms  amplitude  of  acceptable  error 


If  an  operator  is  required  to  monitor  several  such  displays, 
the  v/orkload  associated  with  reading  a set  of  such  instruments 
may  be  estimated  by  deriving  the  information  load  of  each  one  and 
relating  the  sum  of  individual  loads  to  an  ^prdori^  estimate  of 
the  operator's  channel  capacity  (see  Senders, 1964 ) . 


Input  Parameters 


In  order  to  compute  H,  the  probability  distribution  over  the 
stimulus  set  is  required.  It  may  be  known  precisely  or  estimated. 
In  order  to  predict  response  time,  the  intercept  and  slope,  a and  1 
of  Eg . (2),  are  required.  The  value  of  parameter  a will  depend 

on  the  nature  of  the  response  required . rf  no  overt  movement  is 
required,  a --  200  msec,  will  provide  an  adequate  approximation. 

The  parameter  b depends  on  the  extent  of  practice  and  on  the 
stimulus-response  compatibility  of  the  action  to  be  undertaken. 

In  the  absence  of  more  detailed  informal  ion,  a slope  of  .33  soc/bit 
or  a 3 bit  per  sec  information  rate  may  be  assumed  as  a rule  of 
thumb.  Teichuor  and  Krebs,  (1074)  summarizes  parameter  values 
for  a wide  variety  of  laboratory  situations. 
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Mod  o 1 Ou  t p lit  s 

The  model  as  presented  here  predicts  response  time.  However, 
a unique  feature  of  the  information  measure,  is  its  implicit 
assumption  of  a relation  between  speed  and  accuracy  of  performance. 
For  a given  set  of  conditions  and  at  a given  level  of  practice, rate 
of  information  transmission  has  been  shown  to  remain  relatively 
constant;  thus, if  the  operator  increases  his  speed  of  performance, 
the  transmitted  information  decreases  proportionately.  The  compu- 
tation of  information  transmitted  requires  detailed  information 
about  the  distribution  of  responses  and  errors.  If  those  data 
were  available  a priori,  predictions  of  information  rate,  a measure 
combining  spaed  and  accuracy  would  be  possible,  but  usually  these 
data  are  only  available  after  test. 


Model  Validation 


The  basic  relation  between  information  and  response  time  has 
been  extensively  validated  in  the  laboratory,  (see  Teichner  and  Krebs , 
1974).  In  one  application  (Gross,  et  al,  1964)  these  methods 
together  with  other  models  for  prediction  of  movement  times  derived 
from  MTM  principles  were  used  to  predict  workload  for  one  phase  of 
the  Mercury  spacecraft  mission.  Predictions  were  within  10-15%  of 
values  obtained  from  live  mission  simulations. 


Comments 

Although  the  popularity  of  the  information  metric  among  the 
research  community  fourished  in  the  1950's  and  early  GO's  and  has 
decayed  away,  its  significance  as  a summary  measure  of  that  permits 
predictions  of  response  time  in  practical  situations  remains 
important.  The  data  base  from  which  to  select  parameter  values  if. 
large  and  useful. 
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V i ok  or  s'  Accumulator  Mod  - * 1 of  D i scr iminat ion 


Re ferencos 


Vickers,  D.  Evidence  for  an  Accumulator  Model  of 
Psychophysical  Discrimination,  Ergonomics , 13 , 
1970,  p.  37-58. 


De scription  of  Model 

Building  on  considerable  previous  work  on  modelling 
discrimination  accuracy  and  latency/  Vickers  has  assembled  the 
most  comprehensive  model  of  discrimination  performance  available. 

It  is  representative  of  a larger  class  of  more  special  purpose 
models  and  conceptual  frameworks  that  formulate  choice  latencies 
as  a statistical  decision  process  in  which  the  observer  samples 
information  from  the  stimulus  display  and  accumulates  evidence 
over  time  that  increases  his/her  confidence  in  choosing  a 
particular  response.  When  the  evidence  favoring  one  of  the 
alternatives  exceeds  a critical  level  of  confidence,  c,  in  favor 
of  one  response  then  that  response  is  executed. 

More  precisely , the  model  has 'been  formulated  for  a t o- 
alternative  discrimination  process  in  which  an  observer  makes 
comparisons  between  the  noisy  excitation  generated  by  stimulus  A 
havina  a mean  A and  variance  of:  and  stimulus  having  excitation 
mean  3 and  variance  og.  The  distribution  of  differences  is 
converted  to  standard  score  form 

z = (A-B)//(o2  + a2) 

cl  D 

and  partitioned  by  use  of  cumulative  norma]  statistics  into  the 
probr.o  i i ity  p that  A > B and  the  probability  cj  that  B > A (where  q=l-p) 

The  representation  then  departs  from  the  usual  statistical 
decision  formulation  by  postulating  that  the  observer  samples  a 
difference  magnitude  associated  with  each  sample  and  the*  these 
magnitudes  are  also  normally  distributed  random  'mri  tides , 
having  me  t;  m and  n for  positive  ( .V  B ) and  negative  (B  A) 
d i f for : neon . Thus  with  piobability  p the  observer  samples 
posit  i • -e  difie.’encc  having  magnitude  m. 
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The  sampling  process  is  considered  to  be  periodic  and  serial, 
each  sample  taking  a constant  tina,  A.  Thus  the  mean  response 
latency  is  given  by 


L = A N + t 

o 

where  t is  a constant  added  to  account  for  relatively  fixed  delays 
in  apparatus  and  human  response  systems  and  N is  the  mean  number  of 
samples  required  to  exceed  the  response  criterion  Ca  or  Cjj 

i 

The  author  argues  that 


lp  (ra  + l , 


where  r 


and  Ip  (r  , r,  ) is  the  incomplete  beta-function, 
a d 


Input  parameters 

To  generate  relative  latency  distributions  requires  estimates 
of  the  mean  and  variance  of  the  stimulus  excitation  distributions, 
which  may  be  expressed  in  terms  of  the  detection  parameter  d'  or 
as  means  and  variances  directly.  These  parameters,  in  principle, 
should  be  observer  independent,  but  probably  are  not.  The 
parameters,  A and  t must  be  estimated  to  calibrate  the  theoretical 
distributions  to  empirical  latency  distributions.  Finally,  one  must 
estimate  and  C^,  the  values  of  N at  which  the  sampling  process 
is  to  terminate.  Usually  these  would  be  chosen  symmetrically  and 
might  well  be  set  a priori  if  enough  experience  with  their  use  had 
been  accumulated. 

Model  Outputs 

The  basic  output  of  the  model  is  the  distribution  of  response 
time  as  a function  of  the  response  probability  given  that  signal  A 
or  B occurred.  This  amounts  to  the  joint  function  of  latency  and 
probability  of  a correct  response. 
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Model  Va  1 id  at  Lori 

The  eiuthor  has  shown  that  his  model  fits  a variety  of  sources 
of  laboratory  data  better  than  alternative  statistical  decision 
formulations  and  in  fact  resolves  some  unexplained  discrepancies 
between  previous  accumulator  models  and  random  walk  models.  To 
our  knowledge  it  has  never  been  tested  in  a practical  context. 


Comments 

This  model  is  typical  of  those  found  in  the  research  literature 
It  is  more  highly  developed  than  many;  however, the  mathematical 
formulation  deals  only  with  the  two-a Iternative  case.  It  is  likely 
that  it  could  be  generalized,  at  least  in  a computer  simulation 
version, to  multiple  alternatives,  but  then  new  validation  studies 
would  be  needed.  The  principles  of  psychophysical  discrimination 
underlie  many  decision  and  judgment  tasks  in  application 
environments  and  in  that  sense  represent  an  important  basic  functic 
to  be  modelled. 
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Atkinson  and  Shiffrin  Model  of  Human  Memory 


Ho : ev,onc:' 


Atkinson,  R.  C.  , Brel  :•  ford , J.  VI.,  Jr.,  and  Shiffrin,  R.  M. 
"Multi-process  Models  f r Memory  with  Applications  to 
a Continuous  Present, at J on  Task."  J.  Math.  Psych.,  A , 
277-300,  1967. 

Atkinson,  R.  C. , and  Shiffrin,  R.  M.  "Human  Memory:  A Proposed 

System  and  Its  Control  Processes."  In,  K.  W.  Spence  and 
J.  T.  Spence  ( eds . ) , T h ? Psychol  o g.v  o f b e a r-  n 1 n r 1 and  Motiva- 
tion, Yol . 2,  New  York:  Academic  Press  196$,  89-195 • 


Descript i on 


Atkinson  and  Shiffrin  have  proposed  a model  of  human  r.-.-r;.- 
try  which  distinguishes  permanent  memory  structures  from  modi- 
fiable, task  dependent  control  processes.  The  structural  com- 
ponents of  the  system  consist  of  multiple  sensory  registers, 
an  auditory-verbal-linguistic  short-term  store  (STS),  and  a 
long-term  store  (LT3).  Incoming  sensory  information  is  aut  o- 
rr.aoically  registered  in  the  appropriate  sensory  register,  from 
v;hi ?h  it  decays  rapidly  (e.g.,  within  several  hundred  milli- 
seconds in  the  visual  sensory  register).  Under  the  control  < f 
modifiable  attentional  and  scanning  processes,  selected  infor- 
mation can  be  transferred  from  the  sensory  register  to  th  ■ S.’S, 
the  active  working  memory.  The  STS  is  characterized  by  a 
limited  informal!  it  capacity,  estimated  to  be  on  the  order  of 
I«- 5 items*  based  on  several  of  Atkinson  and  Shi f frin  1 s experi- 
ments. An  item  of  information  is  subject  to  decay  from  STS  in 
ab  ■ it  ••  "1  se  ■■  a Is  i • 1 h absence  of  a rehears  il  si  rat  e y. 
Rehearsal  is  an  active  process  in  v/h ich  items  are  retain*  1 5 
a t . 'Tor  of  1 imtted  si.,  and  sys*  emaiically  r-  •nev.a-.i . N .-w  i tor.:; 
car.  dir  place*  old  it,,  -ns  in  th"  rehear:;,)]  buffer,  under  voiun- 
t c ’t:  rol  . The  LTS  is  the  permanent  memory  .‘structure  in  the 

s’.  • ••  . Th  "strength"  >f  the  retires  ntation  of  an  1 1 ••••  in  the 
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• i r i ■'  \ currently  residing  in  the 
r -v  -i  with  probability  1.  Thun, 

•'  retrieving  an  item  correctly  in  given 
item  is  currently  in  STS  plus  the 
T retrieving  it  from  LTS  with  the 
- i n STS . 


In : 5 ut  Pa ramet, e r s 

There  are  two  parameters  which  describe  the  transfer  of 
information  into,  and  out  of,  the  LTS: 

a = the  rat e of  transfer  of  information  about  an 
item  from  STS  into  LTS. 

•r  = the  proportion  of  information  retained  in  LTS 
per  unit  time  ( 1 — v = proportion  of  information 
lost  per  unit  time). 

If  the  rehearsal  buffer  is  hypothesized  as  part  of  a 
person’s  strategy  for  a given  task,  there  are  up  to  three 
other  parameters  which  may  be  specified: 

y = size  of  the  rehearsal  buffer 

a = probability  that  a new  item  (not  currently  in 
the  buffer)  will  enter  the  buffer,  displacing 
an  old  item. 

& = pa  ram- •■‘••a*  describin'.-;  bias  towards  displacing 

oldest  ? temn  in  buffer  before  newest  items. 
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Comment  s 


Th-  strength  of  this  model 
structure  of  a rehearsal  buffer 


is  its  specification  of  the 
and  1 1.;;  related  coni r i 1 pro- 


cesses as  a primary  strategy  in  the  use  of  short-torn,  memory. 
The  mod  -1  stresses  the  role  that  may  be  played  by  the  long- 
term  store  in  what  would  otherwise  be  considered  purely  short- 
term. rm.-mory  tasks.  On  the  other  hand,  the  model  does  not  spec- 
ify either  the  structure  or  the  organization  of  information  in 
the  1 on g-te rtn  store.  Thu.-.,  the  node!  is  primari  Ly  us<  ful  as  a 
descripi  ion  of  human  performance  in  remembering  and  retrieving 


s e duences 
rehears a 1 
proc  foses 


of  items  in  real  time,  where-  shor-t -term  memory  and 
processes  are  Important,  and  coding  and  organ ira t. :i  n 
ar  e n o t imt  o r t an  t . 
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Movement  Index  of  Difficulty:  Fitts'  Law 
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Description  of  Model 

The  model  predicts  the  time  required  to  make  a speeded, 
simple  positional  movement  given  the  distance  to  be  moved  and 
the  accuracy  constraints  on  the  movement.  It  is  basically  an 
empirically  derived  law,  although  the  form  of  the  model  may  be 
derived  from  information  theory  principles  or  feedback  control 
principles. 

Fitts'  basic  equation  is 

2 A 

MT  = a + b log10  ^ 

where  MT  is  the  required  movement  time;  A is  the  distance  to  be 
moved  to  the  center  of  the  target  area;  W is  the  width  of  the 
target  area;  and  a and  b are  empirically  derived  constants. 

The  model  predicts  movement  time  directly,  and  deals  with 
accuracy  implicitly  through  the  definition  of  W.  It  has  been 
shown  that  the  basic  relationship  is  valid  even  when  speed  of 
movement  is  manipulated,  as  long  as  effective  target  width  is 
calculated  using  the  "Crossman  Correction"  (Welford,  p.  147). 

This  correction  assumes  a Gaussian  distribution  of  hits,  and 
defines  the  effective  target  width  as  that  which  includes  96  per- 
cent of  the  hits.  In  most  applications,  however,  the  target  size 
is  predefined  and  movement  time  is  the  dependent  variable. 

While  this  model  is  very  robust,  accounting  for  90  to  95 
percent  of  the  variance  in  movement  times  under  simple  movement 
conditions,  revisions  of  the  equation  have  been  proposed  by 
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Welford  to  achieve  greater  precision  in  a variety  of  conditions. 
His  modification  is  given  by 

MT  = b l°g10  ^ + (b-a)  log^C 

where  A,  W,  a,  and  b are  defined  as  before  and  C is  interpreted 
as  a minimal  scatter  of  target  hits  at  a point  target. 


Input  Parameters 


Movement  distance  (A)  and  target  width  (W)  are  required 
input  parameters.  The  constants  a,  b,  and  C are  assumed  to  be 
relatively  invariant.  Welford  assigns  them  values  of  .104  sec/bit, 
.179  sec/bit,  and  31  mm.,  respectively.  In  Fitts'  formulation, 
typical  values  for  discrete  movements  are  a = -.570  sec  and 
b = .074  sec/bit. 

While  these. parameters  appear  to  be  relatively  invariant, 
they  should  be  checked  empirically  if  the  application  is  to  other 
than  simple  positional  movements  of  a finger  or  stylus. 


Model  Outputs 

In  the  form  shown,  the  dependent  variable  is  movement 
time.  As  discussed  above,  however,  the  model  could  also  be  used 
to  predict  movement  accuracy  as  a function  of  speed. 


Model  Validation 


The  Fitts'  law  methodology  has  been  applied  in  a large 
variety  of  settings,  ranging  from  the  control  of  hand  movements 
under  microscopic  magnification  to  positioning  of  a light  spot 
directed  by  head  movements  in  an  application  to  a photocell- 
operated  typewriter  for  paraplegics.  There  is  even  some 
indication  that  the  basic  equation  is  applicable  to  foot 
movements,  but  with  different  parameter  values.  It  also  has  been 
shown  to  work  for  positioning  pegs  into  holes  and  when  significant 
weight  is  attached  to  the  hand.  To  a first  approximation,  it 
will  work  for  fore-aft  movements  as  well  as  side-to-side 
movements,  although  most  available  data  were  obtained  for  the 
latter  case.  It  does  not  predict  accurately  when  the  ratio  of 
movement  distance  to  target  size  is  less  than  about  2 to  1 . 
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Comments 


This  model  is  potentially  applicable  to  any  task  or  sub- 
task in  which  the  layout  of  positions  on  a work  place  is  relevant 
to  the  design  variables  under  stud"  and  where  movement  time  will 
be  a significant  proportion  of  the  total  time  involved  in  a 
particular  task. 
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Intuitive  Statistics 


NOTE : Like  the  work  on  heuristics  summarized  in  Abstract  No.  29, 

laboratory  efforts  to  evaluate  man  as  an  intuitive 
statistician  and  to  compare  his  performance  with  that  prescribed 
by  normative  statistical  theory  have  potential  relevance  to 
descriptive  modelling  of  behavior  in  operational  settings. 

Below  we  present  a summary  of  the  classic  paper  by  Peterson  and 
Beach  that  highlights  those  aspects  of  human  performance  that 
are  thought  to  be  of  importance  and  that,  with  some  further 
research,  could  yield  to  quantitative  expression. 
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Description 


The  long  range  goal  of  the  work  reported  is  the  development 
of  a theory  of  human  behavior  in  uncertain  environments.  Of 
prime  interest  are  those  environments  that  can  be  characterized 
statistically  and  those  tasks  for  which  a model  of  ideal  per- 
formance can  be  specified.  For  our  purposes,  the  cited  paper 
provides  a convenient  catalog  of  observed  performance  in  two 
major  areas: 

1)  estimates  of  the  characteristics  of  samples  of  data 

2)  the  use  of  samples  to  infer  characteristics  of  parent 
populations 

The  first  of  these  is  referred  to  by  the  authors  as  "Intuitive 
Descriptive  Statistics";  the  second  as  "Intuitive  Inferential 
Statistics . " 
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I .  Intuitive  Descriptive  Statistics 

1.  Judgments  of  proportion:  the  relation  between  mean 

estimates  and  sample  proportions  is  described  well  by 
an  identity  function  with  a maximum  deviation  of  the 
mean  estimate  from  the  sample  proportion  of  0.3  - 0.5 
and  an  average  deviation  very  close  to  zero.  There  may 
be  a very  slight  overestimation  of  low  proportions  and 
an  equally  slight  underestimate  of  high  ones,  depending 
upon  experimental  conditions. 

2.  Judgments  of  means  and  variances:  subjects  familiar 

with  the  concept  of  a mean  of  a distribution  give 
highly  accurate  estimates,  though  there  is  some  variance 
as  a function  of  sample  variance,  sample  size  and  rate 
of  presentation  of  data.  There  are  systematic  discrep- 
ancies between  intuitive  judgments  of  sample  variance 
and  actual  variance.  In  general,  as  means  increase, 
estimates  of  variance  decrease. 


II.  Intuitive  Inferential  Statistics 


1.  Inferences  about  proportions:  in  experimental  tasks 

requiring  sequential  revision  of  subjective  probabilities, 
it  is  typically  found  that  subjects  are  conservative  in 
their  behavior,  revising  their  estimates  in  the  appro- 
priate direction  as  additional  data  are  acquired,  but 
at  a rate  considerably  less  than  is  justified. 

2.  Inferences  about  means:  inferences  concerning  the 

central  tendencies  of  symmetrical  distributions  are 
generally  accurate.  In  J-shaped  distributions , inferences 
concerning  median  and  mode  are  accurate  but  those  relating 
to  the  mean  are  biased  toward  the  median. 

3.  Inferences  about  correlations:  evidence  suggests  that 

subjects  do  not  pay  attention  to  all  cells  of  a 2 x 2 
contingency  table  when  inferring  correlation.  That 
cell  in  which  favorable  outcomes  appear  toqether  or 
those  associated  with  the  positive  diaqonal  seem  to 
receive  primary  consideration.  As  the  tabular  dimensions 
increase  beyond  2x2,  inferences  become  more  nearly 
like  those  assessed  by  normative  correlation  statistics. 
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4.  Determining  Sample  Size:  in  experiments  in  which  a 

subject  must  infer  which  of  two  populations  is  being 
sampled,  he  has  continuous  control  over  the  number  of 
samples  taken,  a cost  is  assigned  to  each  sampled  datum, 
and  the  dependent  variable  is  the  number  of  data 
purchased  prior  to  making  a decision,  it  is  typically 
found  that  although  performance  is  influenced  in  the 
proper  direction  by  cost/payoff  structure,  the  magnitude 
of  the  influence  of  this  structure  is  less  than  that 
prescribed  by  ideal  models.  When  the  ultimate  size  of 
the  sample  is  decided  ahead  of  time  by  the  subject,  the 
cost/payoff  structure  appears  to  have  no  effect  on 
behavior. 

When  prior  probabilities  are  reduced  by  defining  more 
alternatives,  subjects  appropriately  purchase  larger  samples  of 
data.  When  only  two  alternatives  are  involved  and  prior  prob- 
abilities are  made  more  disparate,  fewer  data  are  purchased. 

With  any  number  of  alternatives  or  distributions  of  priors, 
however,  purchasing  performance  is  conservative  relative  that 
dictated  by  the  optimal  (Bayesian)  rule. 


Input  Parameters 


Not  applicable 


Model  Outputs 

Not  applicable 


Model  Validation 


of  performance  summarized  above  were 
conditions  and  most  all  have  been 
and  later  studies  using  a wide  variety 
response  modes.  Efforts  to  define 
purchasing  tasks  within  the  fixed  and 
optional  stopping  paradigms  and  to  construct  descriptive  models 
have  been  particularly  intense  (see,  for  example,  Pitz,  1968, 
1969  and  Pitz,  et  al,  1967,  1969). 


All  of  those  aspects 
observed  under  laboratory 
validated  in  both  earlier 
of  stimulus  materials  and 
parameters  in  in f ormat ion 
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Comments 


In  our  judgment,  observations  such  as  those  above  are  useful 
in  model  building  activity  in  two  different  ways  : (1)  collectively, 

they  help  to  establish  the  forms  of  relationships  between  inde- 
pendent variables  and  performance  in  situations  requiring 
quantitative  inference;  and  (2)  they  help  to  define  the  ranges 
over  which  these  relationships  can  be  expected  to  obtain.  Because 
of  the  frequency  with  which  the  phenomenon  of  conservatism  in 
sequential  sampling  has  been  demonstrated,  we  believe  there  is 
sufficient  justification  for  attempting  to  capture  its  basic  form 
in  descriptive  models  of  human  performance.  For  any  given 
situation,  of  course,  some  prior  empirical  work  must  be  accomp- 
lished to  define  its  range. 

With  respect  to  the  other  indicators  of  sub-optimal 
performance,  we  feel  less  certain.  Perhaps  only  additional 
research  within  the  precise  contexts  of  given  operational  tasks 
can  verify  their  existence  and  significance. 
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Kahneman  and  Tver sky 1 s Heurisl  i ■ 


Mote:  Recently,  psychologists  have  become  interested  in  dis- 

covery and  description  of  the  sorts  of  heuristics,  al- 
gorithms and  rules-of-thumb  people  employ  in  the  solu- 
tion of  problems  requiring  inference  and  reasoning.  At 
this  time,  some  of  the  most  interesting  findings  can 
only  be  described  in  qualitative  terms.  This  summary 
contains  brief  descriptions  of  two  heuristics  which  are, 
in  our  judgment,  of  potentially  great  relevanace  to 
quantitative  modelling  of  human  performance. 
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The  Representativeness  Hypothesis 

According  to  this  hypothesis,  people  select  or  order  outcomes 
on  the  basis  of  the  degrees  to  which  these  out  tones  represent  the 
essential  features  of  the  evidence  they  have  examined.  In  doing 
so,  they  ignore  the  prior  probabilities  of  those  outcomes  and/or 
the  degree  of  unreliability  inherent  .in  the  evidence.  Where  t.h  • 
representativeness  of  the  outsomes  is,  in  fact,  met  died  to  the 
likelihoods,  this  behavior  leads  to  a statistical ! ;;  appropriate 
selection  (or  ordering) . In  situations  where  they  are  not  matched, 
however,  the  selection  (ordering)  will  be  inappropriate . 


An  important  prediction  from  this  bypath, 
the  processes  of  evaluating  a sot  of  data  and 
outcome  on  the  basis  of  the  iat.a  are  independ 
tail  differing  amounts  of  uncertainty  when  th 

perfectly  reliable,  people  will  perf s If 

uncertainties  are  similar.  The  essential  d i 
evaluating  an  input,  and  predicting  an  out.com  ' 
an  example  provided  by  the  authors : 


■ s i s i 3 t ha t wh areas 
of  predicting  an 
■nt  and  should  en- 
iutn  are  1 oss-then- 


:.h  • processes  and 
eron.ee  between 
i high!  ighte'd  in 
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"Suppose  one  Is  told  that  a college  freshman  has  been 
described  by  a counsellor  as  intelligent,  self-con- 
fident, well-read,  hardworking,  and  inquisitive. 

Consider  two  types  of  questions  that  might  be  asked 
about  this  description:  (a)  Evaluation : How  does 

this  description  impress  you  with  respect  to  academic 
ability?  What  percentage  of  descriptions  of  freshmen 
do  you  believe  would  impress  you  more?  and  (b)  Pre- 
diction : What  is  your  estimate  of  the  grade  point 
average  that  this  student  will  obtain?  What  is  the 
percentage  of  freshmen  who  obtain  a higher  grade  point 
average? " 

The  authors  point  out  that  there  is  surely  greater  uncer- 
tainty concerning  the  second  question  than  the  first,  hence, 
that  the  prediction  should  be  closer  to  fifty  percent  than  the 
evaluation.  They  predict,  however,  (and  later  demonstrate) 
that  subjects  will  produce  approximately  the  same  estimate 
for  both  questions  since  they  will  consider  the  best  pre- 
diction to  be  that  scoi’e  which  is  most  representative  of 
the  input  data. 

A second  prediction  resulting  from  the  representativeness 
hypothesis  is  that  the  degree  of  confidence  one  has  in  his  pre- 
diction from  a set  of  data  reflects  the  degree  to  which  the  out- 
come he  has  selected  is  more  representative  of  the  set  than  are 
other  possible  outcomes.  Thus,  one  might  have  more  confidence 
in  the  prediction  of  an  overall  3 average  in  fuuure  cours  :s  on 
the  basis  of  B grades  in  two  separate  introductory  courses  than 
he  would  have  given  an  A and  a C in  those  courses.  This,  one 
authors  note,  is  incompatible  with  the  common  multivariate  model 
of  prediction  in  which  predictive  accuracy  is  independent  of 
within-prof ile  variab ility . 

The  tendency,  if  it  exists,  to  predict  solely  on  the  ban i s 
of  the  input  data  and  to  ignore  the  prior  probabilities  of 
chosen  outcomes  may  also  be  at  the  root  of  commonly  observ  • 1 
failures  to  appreciate  regression  phenomena . In  the  authors' 
view,  the  regression  effect  typically  violates  the  intuition 
that  a predicted  outcome  should  bo  maximally  reoresentut iv > of 
the  input  information. 

The  Avail abi llty  Heuristic 

With  this  heuristic,  a person  judges  the  ‘‘’requenny  or  prob- 
ability of  an  event  or  class  of  events  on  the  basin,  of  th  ease 
with  which  relevant  instances  are  bmu  ht  to  mind.  An  exa  pi  ■ 
provided  by  the  authors  in  thal  >f  ass<  ssing  tl  ■ risk  of  rt 
attack  among  middle-aged  people  by  recalling  such  occurra-n  -•  n. 
among  acquaintances . 
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As  in  the  case  of  inference  via  representativeness,  the 
predictions  one  is  led  to  under  this  heuristic  are  frequently 
correct,  since  instances  of  frequent  events  are  typically  re- 
called better  and  more  quickly  than  instances  of  less  frequent 
events.  Associated  with  its  use,  however,  are  certain  biases 
which  occasionally  produce  inappropriate  j udgments : some  of 

these  are  as  follows: 

(1)  Biases  due  to  retrieved ility  of  instances.  A class  whose 
distances  are  easily  recalled  will,  using  this  heuristic, 
appear  more  frequent  than  a class  of  equal  frequency  whose 
instances  are  less  easily  recalled. 

(2)  Biases  due  to  effectiveness  of  search  set.  In  tasks  re- 
quiring estimates  of  the  relative  frequencies  of  different 
words,  the  availability  heuristic  will  lead  to  the  judg- 
ment that  abstract  words  (e.,g.,  love)  are  more  numerous 
than  concrete  words  (e.g.,  story). 

(3)  Biases  of  imaginability . In  instances  in  which  one  must 
assess  the  frequency  of  class  whose  instances  are  not  stor- 
ed in  memory  one  may  identify  a rule  or  algorithm  that  can, 
in  turn,  generate  instances  for  consideration.  The  ease 
with  which  instances  can  be  generated  then  become  the 
criterion  for  frequency  assessment.  Depending  on  the  char- 
acteristics of  the  rule  and  the  prior  likelihoods  of  the 
instances  generated,  the  assessment  may  or  may  not  be 
accurate. 

(4)  Illusory  correlation.  This  term  refers. to  a tendency  to 
the  frequency  of  a co-oc.curence  of  events  which  are  natur- 
ally associated  with  each  other. 


Input.  Paramete rs 

Not  applicable. 


kodei  Outputs 

We  have  outlined  a number  of  th  • empirical  expectations 
that  follow  from  the  hypotheses  that  representativeness  and 
availability  heuristics  frequently  underlie  the  evaluation  of 
data  and  the  prediction  of  outcomes. . These  and  others.  ■ ■'  per- 
haps lesser  importance  in  the  current  context  can  be  summari:  >d 
as  follows: 


(1)  Judgments  of  the  probab 
to  be  representative  of  the 
j r i ■ > ■ prol  tl  ilit  ies  ansocia 


5 l ilies  of  futur.  events  wi  11  lend 
input,  data  and  not  to  reflect,  the 
■ i with  • hose  events. 


2 \t 


(2)  Judgments  as  the  probability  of  obtaining  a particular 
result  in  a sample  drawn  from  a specified  population  will  be 
relatively  insensitive  to  the  size  of  the  sample. 

(3)  The  confidence  one  has  in  a prediction  will  depend  almost 
completely  on  the  degree  to  which  that  prediction  is  representa- 
tive of  the  input  data  and  will  be  insensitive  to  the  reliabil- 
ity of  the  data. 

( -i ) The  normal  tendency  for  the  performance  of  people  (or 
machines)  to  oscillate  about  some  mean  level  - hence,  for  a 
better-than-average  or  poorer-than-average  performance  to  be 
followed  immediately  by  an  "average"  performance  will  fall  to 
be  recognized.  One  possible  outcome  of  this  failure  may  be  that 
punishment  will  be  perceived  to  be  more  effective  than  reward. 


(5) 

ity 


When  judging  the  relative  size  of  classes  by  the  availabil- 
of  their  instances,  a class  whose  instances  are  easily  re- 
called will  appear  larger  than  one  whose  instances  are  less 
easily  recalled. 


(6)  If  the  frequency  of  words  is  judged  by  availability  of  con- 
, abstract  words  will  be  judged  to  be  more  numerous  than 
ete  words. 


u c: 


(7)  The  frequencies  of  co-occurrences  of  natural  associc 
to  be  overestimated. 


tend 


(8)  The  frequency  of  conjunctive  events  will  tend  to  be  over- 
estimated; the  frequency  of  disjunctive  event: 
underestimated . 


will  tend  to  be 


Model  Vali da t ion 

The  representativeness  and  availability  hypotheses  were 
formulated  on  the  basis  of  empirical  studies  in  a wide  variety 
of  areas  including  probabilistic  inference,  evaluation  of  student 


profile; 


recall  of  materia 


itorea  in  memory,  e? 


sp  :iflc  i dictions  (discussed  under  "Model  Outputs’*  abov 

later  examined  in  carefully  controlled  laborat  >ry  stud  Lor. 
authors.  Those  identified  have  been  found  to  be  valid  ov 
wide  range  of  evaluation  and  prediction  contexts,  and  sor  • 
the  ) v resti ma tion  of  events  having  a natural  assoc ial  i > ) 
remarkably  resistant  to  contradictory  data. 


The 

••)  w e r • 
by  the 
r a 
( • ' • r . , 

seen 


common 


As  ■ i In  the  pn  face  to  this  summary,  w fe 

■ ' type  have  pol  ;n i : a ' 1 ea  1 relevai  se 

‘ v ■ ! • 1 1 ' a - of  hut  •••!  |u>tT  Th  v waul 
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be  applicable  to  many  situations  in  which  an  operator  must  pre- 
dict the  future  state  of  a system  on  the  basis  of  noisy  input 
data,  or-  where  fie  is  required  to  perform  fault  isolation  and 
trouble  shooting  tasks.  We  expect  that  for  such  situations, 
the  basic  forms  of  the  relationships  between  input  data  and 
evaluation  and  prediction  performance  would  be  relatively  easy 
to  determine  empirically,  though  values  suitable  for  precise 
estimation  in  either  analytic  or  simulation  models  might  require 
a relatively  extensive  program  of  parametric  research. 

The  point  should  also  be  made  that  since  the  behavioral 
phenomena  of  interest  relate  largely  to  man's  capabilities  as 
an  intuitive  statistician,  there  exists,  in  the  form  of  pre- 
diction from  normative  statistical  theory,  a ready  standard 
against  which  to  compare  to  his  performance.  The  ability  to 
achieve  this  comparison  can  aid  directly  In  assessing  the  rela- 
tive merits  of  alternative  system  aids  to  the  evaluation  and 
prediction  performance  of  the  human  controller. 


2 3k 
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The  Bayesian  Model  of  Inference 
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Description 


Given  a set  of  mutually  exclusive  and  exhaustive  hypotheses, 
(Hi)  and  datum,  D,  Bayes'  rule  expresses  the  probability  that  a 
given  hypothesis,  Hj,  is  true  as  a function  of  p(DjHj),  the 
probability  that  D will  be  observed  given  that  Hj  isJtrue,  and 
p ( H j ) , the  probability  that  Hj  is  true  prior  to  the  observation 
of  D. 


P(D|h.)p(H.) 

P (H  . | D)  = 3 1_ 

p(D) 

where  p(D)  = ~ p (D  | II . ) p (II  . ) n = total  number  of 

1 1 hypotheses  in  Oh) 

When  a sequence  of  observations  is  made,  this  formulation  is 
applied  recursively  and  the  value  of  p(Hi|D)  that  is  computed  as 
the  result  of  one  observation  is  employed  as  the  p(Hy)  for  the 
next . 


Since  Bayes'  rule  specifies  an  optimal  policy  for  aggrega- 
tion of  data  against  hypotheses  and  for  the  sequential  revision 
of  probabilities  distributed  over  alternative  hypotheses  as  data 
are  examined,  it  has  served  as  an  important  normative  framework 
with  which  to  evaluate  the  decision  making  performance  of  real 
decision  makers.  Further,  it  has  been  advanced  as  a useful 
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decision  aid  in  the  operation  of  complex  man/machine  data 
acquisition  and  data  processing  systems,  particularly  those 
dealing  with  intelligence  information.  Recently,  application 
of  the  rule  has  been  generalized  to  situations  where  the 
reliability  of  data  may  be  suspect  (see  Abstract  Mo.  31  ) . 


Input  Parameters 

Exploitation  of  Bayes'  rule  requires  that  the  prior  prob- 
abilities of  each  of  the  possible  hypotheses  be  input  and  that 
the  set  of  conditional  probabilities  [p(D|Hj)]  be  defined.  As 
indicated  above,  the  priors  [ p ( II j ) ] need  only  be  defined  for  the 
first  iteration  in  a sequential  revision  situation. 


Model  Outputs 


Bayes'  rule  provides  an  optimal  estimate  of  the  conditional 
probability  p(Hj_|D)  after  the  examination  of  each  new  datum. 


Model  Validation 


Comparison  of  man’s  actual  inferential  performance  against 
the  dictates  of  Bayes'  rule  has  been  one  of  the  leading  pre- 
occupations in  modern  decision  making  research  and  theory. 

These  efforts  have  been  directed  at  finer  understanding  of  one 
of  the  earliest  findings  in  the  Bayesian  decision  making  area, 
viz . that  humans  appear  to  assign  a less  extreme  probability  to 
the  likelihood  of  a given  hypothesis  after  examining  a giver 
datum  than  is  justified  on  the  basis  of  the  prescriptive  rule. 

Despite  the  acknowledged  productivity  of  this  framework, 
there  are,  in  our  judgment,  a number  of  limitations  that  must  be 
considered  in  assessing  the  generality  and  utility  of  the  rule 
to  real  decision  making  situations.  Our  concerns  are  dealt  with 
at  some  length  in  Ref.  2 above  and  will  only  be  summarized  here. 

1)  The  rule  concerns  only  that  aspect,  of  decision  making 
related  to  hypothesis  evaluation. 

2)  It  requires  uncertainty  to  be  expressed  in  terms  of  a 
mutually  exclusive  and  exhaustive  set  of  hypotheses. 


2 ;o 
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3)  New  hypotheses  cannot  be  added  to  an  existing  set  of 
hypotheses  once  evaluation  has  begun. 

4)  Bayes'  rule  has  nothing  with  the  "truth  value"  of  a 
given  hypothesis,  only  with  its  statistical  support. 

5)  Incorrect  assignment  of  prior  likelihoods  may  bo 
devastating  when  hypotheses  are  evaluated  on  the  basis 
of  few  data. 

6)  Bayes'  rule  does  not  provide  a direct  criterion  for 
"stopping"  in  a sequential  data  acquisition  situation. 
Such  a criterion  must  be  adduced  to  the  decision 
framework  from  outside. 


1 
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Abstract  No.  31 


Bayesian  Models  of  Decision  Making  With  Unreliable  Data 

Note : The  following  models  are  presented  because  of  their 

potential  relevance  to  modelling  of  controller  judgments 
as  to  whether  or  not  "noisy"  reports  of  vehicle  position  indicate 
"on-course"  or  "off-course"  behavior.  They  are  presented  as  a 
group  primarily  because  of  their  formal  similarity. 
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Description 

Unlike  most  Bayesian  models  of  human  information  processing, 
this  family  of  models  explicitly  considers  that  the  data  on 
which  the  decision  maker  must  base  his  judgment  have  less  than 
perfect  reliability.  In  the  first  of  the  two  prescriptive  models 
presented  below,  the  source  of  error  arises  during  the  original 
observation  of  an  event  and  the  observer /decision  maker  must 
estimate  whether  or  not  the  event  actually  occurred.  In  the 
second,  the  error  arises  during  the  reporting  of  the  event  to 
the  decision  maker  by  another  observer.  In  this  latter  instance, 
the  report  is  made  without  qualification  and  the  decision  maker 
must  formally  discount  the  impact  of  the  reported  event.  Both 
models  are  of  the  "cascaded  inference"  type,  requiring  that  the 
decision  maker  first  estimate  the  impact  of  the  datum  as  though 
it  were  perfectly  reliable  and  then,  in  a second  step,  alter 
this  nominal  impact. 


1 . Prescriptive  Approaches 
1.1  The  Dodson  Model 

Dodson  (1961)  considered  the  situation  in  which  an  observer 
is  not  certain  which  of  two  mutually  exclusive  events,  D1  and  D , 
has  occurred,  but  may  be  able  to  make  a probability  or  certitude 
judgment  on  the  question.  He  suggested  that  in  order  to  calculate 
the  posterior  probability  of  a hypothesis  in  this  case,  one  should 
calculate  its  value,  given  each  of  the  possible  events,  and  then 
take  a weighted  sum  of  these  values,  the  weights  being  the  proba- 
bilities that  the  observer  attaches  to  the  event  possibilities. 
Given  only  two  possible  data,  the  calculation  may  be  represented 
as  follows: 

UltjD)  = iMD1)p(Hi|D1)  + t ( D 2 ) p ( II  i | D 2 ) 

where  £(th|D)  is  the  posterior  probability  of  II.,  taking  the 
observer's  uncertainty  into  account,  and  ip  (D . ) 1is  the  probability 
that  the  observer  attaches  to  the  possibility  that  he  has  observed 
event  D..  More  generally,  given  n possible  events  and  the  assump- 
tion thilt  the  observer  can  attach  a probability  to  each  of  them, 
the  formula  might  be  written  as: 


UH.  | D ) = T.  t (D  . )p(H.  [ D . ) . 
1 j--l  J 1 


24  3 


Report  No.  3446 


Bolt  Beranek  and  Newman  Inc. 


Substituting  the  Bayesian  formula  for  pCH.JCh),  yields  the  expression 


p (D  . I H . ) p (H  . ) 

UHi|D>  ■ l*' 'V  r • 


1.2  The  Models  of  Schum  and  DuCharrne 


In  this  model,  there  are  two  possible  hypotheses,  H.  and  H , 
two  possible  data  events,  D,  and  D ^ , and  two  possible  reports  of 
the  event  that  occurred,  d^  and  d„7  The  decision  maker's  task 
is  to  determine  p(Ih|dj).  1Accordrng  to  Bayes’  rule: 


P(Hi|aj)  = 


P(d.  |Hi)p(Hi) 

PTdJ)  ~ 


which  can  be  manipulated  to  obtain: 


p(d.  IIP)  = EP(Dk|Hi)p(d;j)HifiDi 


Given  this,  a likelihood  ratio  (A)  adjusted  for  the  diagnostic ity 
of  the  datum  can  then  be  derived. 


P(d.j  |Hi) 

A = P(d  :fnkT 

The  authors  distinguish  four  decision  "cases"  which  differ  with 
respect  to  the  symmetry  of  the  impact  of  observing  a given  event 
on  p (D  | II)  and  p(d|D)  and  derive  the  optima]  A for  each.  A summary 
of  each  of  these  follows: 

Case  I:  Symmetric  p(D|h):  Symmetric  p(d|D) 

P(D1|H1)  = p(D2jH2);  p(dl|D1)  = p (d  2 1 1)2 ) . 


A „ = pr+  (1-p)  (1-r) 
(l-p)rtp(l-r)' 
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where 


p = 

P(b± 

' H i ) 

1-p  = 

P CI3  ■ 

1h±)  , 

jrl 

r ^ 

P(d± 

1°.) 

1-r  = 

P(d  . 

1 D±)  , 

J?1 

Case  II: 

Asymmetric 

PfDjJ 

ll^)  j- 

2 1 2' 


‘2 1 2 


a , s = 


p1r+(l-p1) (1-r) 
P2r+ (l-p2) (1-r) 


or,  equivalently, 


v;here 


A.  = Pl+k 


a , s 


p2+k 


k = V.5. 


1_Pl 


= P(D1|II1) 
5 p(D2|H1) 
= p(D  | H 


1-Po  = 


r 2) 

p(d2|h2) 


and  r and  1-r  are  defined  as  above 


I 

I 


24  3 


Report  No.  3446 


Bolt  Beranek  and  Newman  Inc. 


Cage  lit : Symmetric  p (D | H)  : Asymmetric  p (d | D) 

p(D1|lI1)  = p(D2|H2);  p(d1|D1)  { p(d2|D2) 


A _ pr1+  d-p)  ( l-r2  ) 
r> ' cl  T 1-p)  r^+p  ( 1-r. ~ T 


or  if  p^l  and  r.^1, 


As,a  “"-Uzb! 


+ 3 


c+ 


I £L| 

U-Pi 


where 


JLm 

C = _1 . 

1-r  2 


Case  IV;  Asymmetric  p (D | H) ; Asymmetric  p(d|D) 

P^1lH1)  + piu2|H2);  p(d  |DX)  f p(d2|D2) 


a , a 


_ P.iri'h  (1“Pj  ) (1-r2) 

P2rl+(1"P2) (l_r2) 


or  if  r1  ^ ( l-r2  ^ 


a , a 


_ Pj+b 
p~+b 


where 


b = 


1~r2 
r j - ( 1 -r  2 ) j 


and  the  remaining  parameters  are  defined  cis  above. 


7.  Empirical  Approaches 


In  addition  to  an  inferos t in  developing  prescriptive  models 
against  which  actual  performance  can  be  assessed,  investigators 
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in  this  area  have  derived  a number  of  empirical  models,  the  most 
significant  of  which  are  summarized  below. 

2 . 1  Snapper  and  Fryback 

These  authors  suggest  that  in  dealing  with  unreliable  data, 
decision  makers  first  estimate  the  likelihood  ratio  as  though  the 
data  were  completely  reliable,  then  adjust  this  ratio  by  mul- 
tiplying it  by  the  reliability  of  the  quotient,  and,  finally,  apply 
the  adjusted  ratio  to  the  calculation  of  posterior  odds. 

Stage  1:  compute  A = rL 

Stage  2:  compute  = Aft  . 

where  r = the  reliability  of  the  report 

L = the  likelihood  ratio 

= the  adjusted  posterior  odds 

ft  = the  (unadjusted)  prior  odds 


2.2  Gettys,  Kelly  and  Peterson 

This  model  assumes  that  the  decision  maker  estimates 
posterior  odds  assuming  the  most  likely  event  is  true,  and  then 
adjusts  the  odds  to  reflect  the  reliability  of  the  source. 

State  1:  compute  f.^  = Lft^ 

Stage  2:  compute  = rft^. 


2.3  Edwards  and  Phillips 

These  authors  present  evidence  suggesting  that  posterior  odds  are 
described  by  a single  stage  process,  as  follows: 


where  c varies  with  L. 


Input  Parameters 

As  with  other  Bayesian  models,  initial  odds  and  the  conditional 
probabilities  p(D.|lI.)  are  required  input.  In  addition,  the  value 
of  v is  required  in  the  Dodson  model  and  r is  required  in  all  others. 
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Model  Outpu t s 


The  Dodson  and  Schurn  and  DuCharme  models  provide  optimal 
estimates  of  diagnostic  impact  for  the  cases  each  considers.  The 
empirical  models  provide  descriptions  of  actual  (non-opt.  i r al ) performance. 


Model  Validation 


Research  indicates  that  although  the  behavior  of  decision  makers 
in  choice  experiments  is  clearly  influenced  by  the  stated  reliability 
of  the  reporting  source,  the  performance  is  not  in  accord  with 
prescriptive  formulations.  In  general,  results  suggest  that  subjects 
exhibit  the  classically  expected  conservatism  with  data  of  relatively 
high  reliability  and  considerably  less  conservatism  than  justified 
with  data  of  low  reliability . The  collective  effort  to  construct 
empirical  models  of  the  decision  process  highlights  the  departure 
from  ideal  performance. 


Comments 


As  noted  at  the  beginning  of  this  summary,  this  family  of 
models  may  have  some  application  in  situations where  a decision  maker 
must  process  and  eventually  decide  the  meaning  of  a set  of  noisy 
state  data.  In  tne  RPV  monitoring  task,  such  a situation  may  occur 
where  the  controller  must  decide  whether  or  not  a sequence  of  position 
reports,  known  to  have  some  probability  of  error,  signifies  a departure 
from  planned  course. 

A shortcoming  \'/ith  most  experiments  in  this  area  is  that  subjects 
are  not  usually  apprised  of  the  correct  method  of  adjusting  diag- 
nosticity  of  unreliable  reports.  Their  performance  given  this 
information  should  be  of  interest  in  the  modelling  of  skilled  behavior. 


Repo r t.  No . 3 4 4 6 
Abstract  No.  32 


Bol  L Be  ran 


and  Newman  Tnc. 


Dun lay , et  al . Models  for  ATC  Conflict  Perception 
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Descr iption 


The  goal  of  Dunlay,  et  al's  work  is  the  development  of 
models  of  the  reactions  of  human  air  traffic  controllers  and  of 
the  distributions  of  aircraft  in  controlled  airspace  that  could 
be  employed  in  combination  to  estimate  the  number  of  potential 
conflicts  requiring  controller  intervention,  and,  therefore,  an 
estimate  of  controller  workload.  The  effort  appears  to  be  unique 
in  that  the  authors  consider  not  only  the  separation  at  which 
controllers  should  take  action,  as  defined  by  FAA  separation 
standards,  but  also  the  distribution  of  separations  at  which 
they  have  been  observed  to  take  action. 

Three  explicit  sources  of  error  are  considered  in  connection 
with  the  controller's  perception:  (1)  limitations  on  the  accuracy 

with  which  separations  between  displayed  targets  can  be  judged? 

(2)  limitations  on  the  accuracy  with  which  target  trucks  can  bo 
extrapolated  when  estimating  the  probability  of  future  intersec- 
tion with  other  tracks;  and  (3)  limitations  on  the  accuracy  with 
which  the  radar  targets  themselves  are  displayed . As  th"  authors 
note,  few  data  exist  with  respect  to  the  first  two  of  these  error 
sources.  The  data  employed  to  build  elements  of  the  model  are 
thos  ■ of  Connolly  and  McCasker  (1970),  Mangel:  (1955),  and 

Dav id  (1967). 
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The  consequences  of  these  limitations  are  of  interest  in 
two  different  conflict  situations:  (1)  those  which  occur  when 

one  aircraft  overtakes  another  on  the  same  heading;  (2)  those 
which  occur  when  the  (projected)  tracks  of  aircraft  on  different 
headings  intersect.  The  total  perceptual  error  (r)  is  assumed 
to  be  the  sum  of  two  independent  rand,  mi  variables:  (1)  total 

extrapolated  judgment  error  (e^  ) , and  (2)  radar  system  error, 


r-{ 

Cl) 

11 

U) 

+ C2 

where  ~ ( 

1 N (-0. 2 
| N(-0.5 

nmi . , 
nmi  . , 
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1 
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1 
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0.4 

1 

nmi. ) • 

and  the  notation  N(a,b)  is  interpreted  as  a normal  distribution 
with  mean  a and  standard  deviation  b. 

The  probability  of  a conflict  given  an  actual  separation,  S,  is 
given  as 

P { Conflict | S = s}  = P{ (s+e) }<  SD> 

where  SD  is  a decision  threshold  at  which  the  controller  must 
intervene . 

Thus,  the  value  of  Sp  for  which  the  a priori  level  of  per- 
formance (say,  that  a controller  be  able  to  detect  potential 
violation  of  a 5 nmi  rule  957,  of  the  time)  for  overtakes  can 
be  expressed  as 


P{  (5+c)  } 

1 A 
72 

II 

.96 

_SD  “ 

(5-0.1)  " 

= 6.0  5 nni.i. 

0. 

7 
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with  corresponding  conditional  conflict  probabilities, 


P {Conflict | S = s} 


and 


P{Conflict|s  = s} 


P{  (s+c ) < 6}  = < 


P{ (s+e)  < 9}  = 


6-  (s-0.1)  ! 


. 7 


9- (s-0.4) 
2.7 


where  4)  is  the  desired  probability  of  detection  (0.95  in  this 
example)  . 


These  models  are  employed  in  combination  with  these  pre- 
dicting particular  track  relationships  in  a volume  of  air  traffic 
in  order  to  estimate  the  expected  number  of  conflicts  over 
specified  time  periods  and,  as  a result,  the  expected  communica- 
tion workload  of  the  controller. 


Input  Parameters 

Display  resolution,  controller  extrapolation  error  as  a 
function  of  time  into  the  future  and  of  track  angle  (relative 
to  other  aircraft  of  concern),  normative  decision  threshold 
and  desired  level  of  correct  identification  are  required  inputs. 


Model  Outputs 


The  primary  outputs  of  the  model  summarized  above  are 
estimates  of  the  distances  at  which  aircraft  in  potential 
overtake  and  crossing  situations  will  be  perceived  by  the  con- 
troller to  be  in  apparent  conflict,  as  a function  of  specified 
levels  of  desired  performance. 

Model  Valida t ion 

As  noted  earlier,  few  data  exist  with  respect  to  the  types 
of  controller  errors  of  interest  to  the  author.  That  controllers 
sometimes  act  considerably  in  advance  of  the  decision  time 
specified  in  the  regulations  is,  however,  a well-known  ph.- some  non . 
Thus,  the  basic  character  of  the  model's  outpu',  is  valid. 
Validation  studies  of  the  precise  values  output  by  th<>  model 
have,  however,  not  been  accomplished. 
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Comments 

The  authors  have,  in  our  judgment,  done  a creditable  job  in 
using  the  few  data  that  exist  to  model  a complex  phenomenon. 

The  sources  of  error  identified  by  them  and  the  basic  format  of 
the  model  are  intuitively  appealing,  though  attention  might  also 
be  given  to  the  cost/payoff  structure  implicit  in  the  situation 
they  are  attempting  to  model. 

We  believe  that  the  model,  if  valid,  may  be  of  utility  in 
the  modelling  of  RPV  controller  behavior  in  those  situations 
where  RECON  and  ELINT  vehicles  must  be  recirculated  for  purposes 
of  accompanying  STRIKE  vehicles  approaching  nandoff.  It  might 
have  additional  utility  in  modelling  of  patching  performance, 
where  there  is  need  to  extrapolate  from  a given  track  the 
possible  point  of  intersection  with  a desired  track. 
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Description 

Arad's  model  represents  the  first  major  attempt  to  generate 
a general  model  of  air  traffic  controller  workload.  The  original 
application  of  the  model  was  to  identify  optimal  sector  geometries 
to  minimize  overall  workload  for  a given  traffic  volume. 

The  total  load,  L,  imposed  on  an  air  traffic  controller  is 
divided  into  several  components: 

L = L0  + Li  + L2  + L3  + ...  + Ln 

L()  is  a "background  load"  component  that  includes  all  loads 
purely  internal  to  the  system  and  unrelated  to  sector  geometry 
and  size,  treaffic  volume,  or  other  measure, ible  phenomena. 

1.1  is  a "routine  load"  component  associated  with  the  minimum 
control  functions  required  to  move  an  aircraft  into  and  out  of 

a sector  when  no  interaction  with  any  other  aircraft  is  considered. 
This  component  is  directly  proportional  to  the  number  of  air- 
craft in  the  system  per  unit  time. 

1.2  is  an  "airspace  load"  component  involving  expected  numbers 
of  conflicts  of  various  types  per  unit  time  and  the  work  required 
to  resolve  each  typo.  Expected  conflicts  depend  heavily  upon 
traffic  flow  patterns  through  a sector,  and  several  subcomponent:', 
are  analyzed.  Generally,  this  component  is  proportional  to  the 
squire  of  the  number  of  aircraft  transiting  the  sector  per  unit 
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L3  through  Ln  are  "induced  load"  components  that  arise  from 
coordination  requirements  with  adjacent  sectors  beyond  basic 
handoff  procedures  (which  are  included  in  Ig) . 

The  underlying  structure  of  Arad's  model  is  s traight forward , 
and  most  of  the  load  subcomponents  are  developed  from  intuitively 
satisfying  models.  A large  number  of  constants,  parameters,  and 
proportionality  constants  arise,  however,  which  must  be  specified 
before  the  model  can  be  applied. 


To  quantify  the  model  parameters,  Arad  employed  several 
hundred  "snapshots"  of  particular  air  traffic  control  situations, 
which  were  presented  to  air  traffic  controllers  in  pairs.  The 
controllers  were  asked  to  visualize  the  situations  presented 
and  to  indicate  which  of  each  pair  represented  the  higher  work- 
load. The  "snapshot"  pairs  presented  had  been  selected  care- 
fully to  explore  the  effects  ol  certain  variables  and  the 
interactions  between  them.  Statistical  analyses  of  the 
controllers'  cho'ces  were  used  to  quantify  the  parameters. 

The  structure  of  Ratner ' s model  (called  RECEP , for  RElative 
Capacity  Estimating  Procedure)  is  almost  identical  to  that  of 
Arad's  model.  This  model  focuses  more  closely  on  the  times 
required  to  deal  with  the  various  events  that  arise.  Its 
parameters  v/ere  determined  by  videotaping  sequences  of  actual 
air  traffic  control  activity,  replaying  them,  and  asking  the 
controllers  why  they  took  each  action  they  did  and  when  they 
had  decided  to  take  this  action.  Specific  event-processing 
ti.es  were  then  deduced  from  a detailed  time  study  of  the 
annotated  videotape. 


Input  Parameters 

The  geometry  of  the  sector  and  all  airways  within  it  must 
be  specified,  along  with  statistical  descriptions  of  the  air 
traffic  within  the  sector  and  of  potential  conflicts  that  can 
occur . 


Mo  del  _0u  t put 

The  models  yield  scalar  representations  of  overall  expected 
controller  workload  for  each  particular  situation  specified. 
Workload  can  al.  be  determined  as  func' ions  of  particular 
'•■■riabl.es,  but  representations  can  become  complex  very  rapidly 
as  tne  number  of  independent  variables  is  increased. 
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Model  Validation 

The  judgments  of  different  controllers  in  Arad's  "snapshot" 
tests  were  found  to  be  highly  consistent.  The  model  thus 
appears  to  be  internally  consistent,  and  should  be  capable  of 
accounting  for  at  least  some  of  the  discrepancies  that  have 
been  shown  to  exist  between  controllers'  perceptions  of  work- 
load in  operational  situations  and  nominal  workload  measures 
computed  by  simpler,  traffic-counting  procedures.  Unfortunately, 
there  is  no  evidence  available  in  the  literature  surveyed  to 
indicate  that  field  validation  of  the  model  has  ever  been 
attempted . 

! 

! RECEP  model  parameters  were  obtained  from  analyses  of 

field  data,  and  the  model  has  been  used  to  predict  the  effects 
of  automating  certain  ATC  procedures.  Actual  validation  of  the 
model,  however,  was  limited  to  a comparison  of  various  decision 
time  estimates  obtained  in  different  centers. 


Comments 

Despite  its  lack  of  operational  validation,  this  model  may 
provide  a valuable  framework  on  which  to  build  models  of  controller 
workload  in  situations  similar  to  air  traffic  control. 


Report  No.  3 4-16 
Abstract  No.  34 


Bolt  Deranek  £ind  Newman  Inc. 


Sheridan *s  Mode j of  Process  Samp! ing  rr « -guoncy 
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De scr iption 

This  model  sets  forth  a prescriptive  framework  for  determining 
how  often  a supervisor  should  sample  a process  under  his  control 
as  a function  of  the  extent  of  his  control  between  samples  and 
the  cost  of  sampling.  The  model  is  based  upon  Bayesian  concepts 
and  makes  the  following  assumptions: 

1)  The  functional  relationship  between  the  value  (V) 

of  the  system's  independent  input  (X)  and  control  input 
(Y)  .is  known  and  the  supervisor  will  attempt  to  maximize 
the  expected  value. 

2)  A stationary  prior  distribution  of  X exists  and  is 
known . 

3)  After  each  sample  is  taken,  the  parameters  of  the 
distribution  regress  monotonically  toward  those  of  the 
prior  distribution. 

Given  the  general  formulation  of  the  expected  value  of  a control 

input, 

<v|y>  = /< V | XV  > { X } 

k 

where  / is  a generalized  summation  over  all  X,  and  { X } is  the 
a priori  probability  of  X.  Three  specific  cases  are  considered. 
The  first  of  these  is  tbs  case  where  only  the  a priori  distribution 
{ X } of  X .is  known.  The  second  is  where  the  supervisor  has  perfect 
knowledge  of  X (presumably  as  a result  of  continuous  sampling). 

The  third,  and  perhaps  most  interesting,  is  where,  because  of  the 
cost  of  each  observation,  the  supervisor  must  select  a strategy  of 
intermittent  sampl ing/con trol . An  aspect  of  additional  interest  is 
that  in  the  second  and  third  case:.,  a preposterior  analysis  can  be 
conducted  with  respect  to  X and  f X t su  h that  i f an  optimal  Y were 
to  bo  chosen  for  each  X,  <v|Y  • could  b • assessed  and  an  optimal 
super  ol-y  policy  adopted  prior  to  X's  actual  occurrence. 

A final  stop  in  the  development  of  the  model  is  the  addition 
of  cost  -of-obs  : r v a ' ion  and  1 .pi  i nq  interval  parameters  which  enable 
the  selection  of  an  optimal  sampling  interval  (or,  alternatively,  an 
optimal  sampling  frequency). 
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Input  Parameters 


The  prior  distribution,  {x},  of  X,  the  expected  value  relationship, 
<VTjxY>,  and  the  cost  of  an  observation  must  be  input. 


Model  Outputs 

The  model  specifies  the  optimal  time  between  samples  or  optimal 
sampling  frequency  for  the  intermittent  sampling  case.  If  the  case 
where  the  a priori  distribution  is  known  and  the  case  where  perfect 
information  is  available  are  also  analyzed,  the  difference  between 
respective  values  of  <vjY>  can  be  considered  to  be  the  expected  value 
of  continuous  supervision/control. 


Model  Validation 


Because  it  is  prescriptive  in  nature,  no  attempts  have  been 
made  to  validate  the  model. 


Comments 

Although  not  a model  of  human  behavior,  this  framework  may  have 
utility  in  applications  where  one  wishes  to  ascertain  the  impact  on 
system  performance  of  optimal  decision  behavior  under  given  cost  and 
"knowledge"  constraints.  Additionally,  it  appears  useful  as  a reference 
against  which  the  actual  performance  of  controllers  might  be  assessed. 

In  this  latter  context,  it  might  also  be  modified  (possibly  by  the 
introduction  of  non-linear  value  and  memory-decay  functions)  such  that 
it  could  serve  as  a descriptive  model  of  human  performance. 
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Drury's  Model  of  Search  and  Decision  Making 
Processes  in  Materials  T n spec t ion 
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Description  of  Model 

This  model  was  formulated  to  account  for  the  empirical 
finding  that  as  quality  control  inspection  time  increases, 
false  rejections  of  "good"  items  increase  as  false  acceptances 
of  "faulty"  items  decrease. 

The  model  assumes  that  an  inspector's  response  represents 
the  outcome  of  a two-stage  process  that  initially  detects  possible 
flaws,  then  subsequently  compares  them  against  an  implicit  or 
explicit  criterion  in  order  to  reach  a decision  as  to  whether 
candidate  items  should  be  accepted  or  rejected. 

On  the  basis  of  Crossman's  (1955)  data  on  discrirnin- 
ability  and  his  own  research  on  inspection  time,  the  author 
formulates  the  following  model  for  the  probability  of 
accpeting  an  item  as  of  time  t(s) 


(Accept)  - exp  j ) * P(f|a)  ( 1 -exp 


where  t (s)  is  the  time  taken  to  locate  a flaw;  t|(s)  is  a scale 
parameter  dependent  upon  search  area,  contrast,  size,  etc.;  and 
p(f|a)  is  the  probability  of  finding  a flaw  that  is,  in  fact, 
acceptable . 


I 


: v , a 
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The  probability  of  rejecting  an  item  as  of  that  time  is 
cj  i vr  •«.  - n ci  s 


p (Reject)  = p ( f | u ) j 1-expj  — i 


t , ( 3 ] 


where  p(fju)  is  the  probability  of  finding  a flaw  that  is,  in 
fact,  unacceptable;  and  t(s)  and  t2(s)  are  defined  as  above. 

I n put  Parameters 

The  a priori  probabilities  of  acceptably  and  unacceptably 
flawed  items  and  the  criterion  for  flaw  acceptability  are 
required  input  parameters,  as  are  the  values  of  the  scale 
parameters,  t^(s)  and  t^fs). 

Mo  del  0 u t p u t s 

The  model  predicts  that  as  search  time  increases,  the 
probability  of  accepting  a good  item  will  decrease  and  the 
probability  of  rejecting  a faulty  item  will  increase.  Further, 
it  predicts  that  the  overall  inspection  per  item  will  be  equal 
to  the  sum  of  search  time  and  decision  time. 


Model  Va 1 i da t ion 

An  experiment  conducted  by  the  author  involving  a visual 
search  of  flat  glass  yielded  the  expected  exponential  relation- 
ship between  cumulative  probability  of  locating  a flaw  and  the 
time  taken  to  locate  it. 

An  experiment  on  decision  making  indicated  that  as  the 
difference  between  acceptable  and  unacceptable  flaws  decreased, 
decision  time  and  errors  increased.  Less  variation  was  found 
in  time  than  in  error,  however. 


2>9 
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Comments 

The  utility  of  the  model  is  u;  sown  at  this  time.  However, 
it  would  appear  that  the  simplicity  of  the  general  framework  - 
that  is,  the  division  of  the  inspection  task  into  a processes  of 
detection  and  binary  classification  - should  make  it  applicable 
to  a wide  variety  of  monitoring  tasks  where  a decision  rule  can 
be  stated. 

To  the  extent  that  diameter  (or  area)  of  a flaw  is  similar 
to  lateral  deviation  of  a vehicle  from  an  expected  path,  the  model 
might  be  used  to  predict  the  success  with  which  monitors  perform 
in  the  context  of  different  display  scales  and  time  stresses. 
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Thomas'  Model  of  Combined  Manual  and  Decision  Tasks 


References 

Thomas,  M.  U.,  "A  human  response  model  of  a combined 
manual  and  decision  task,"  IEEE  Trans.  Sys. , Man 
and  Cybernetics,  SMC- 3,  Vol.  5 , 1973. 

Thomas,  M.  U.,  "Some  probabilistic  aspects  of  performance 
times  in  a combined  manual  and  decision  task,"  Ph.D. 
dissertation,  U.  Michigan,  Ann  Arbor,  1971. 


Description  of  Model 


The  purpose  of  this  model  is  to  predict  the  time  required 
to  perform  a task  consisting  of  n subtasks,  in  one  of  which  a 
temporally  uncertain  signal  occurs  that  requires  the  subject  to 
choose  one  from  among  m alternative  responses. 

The  model  considers  a repetitive  manual  task  composed  of  N 
subtasks,  Qq . . . , QR.  One  of  these  subtasks,  Qj,  receives 
"input  stimulations"  from  a stationary  random  process  I.  (X)  , 
resulting  in  a requirement  for  a choice  of  responses  from  a 
set,  L. 

Two  input  process  structures  are  modelled: 

(1)  An  aperiodic  Markov  process  described  by  an  L x L 
transition  matrix  with  probabilities 


where  p..  = P (X,  = x.lx  = X.)  i,j  = l,...,L, 

i j k j 1 l - 1 i J 

k = 1,2,... 

(2)  A non-Markov  process  with  discrete  probability  function 

p = (px  , . . . , PL) 
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where 

Pj  - P{Xk  = Xj]  = P{Xr  = = x.} 

i I j ~ 1 I • • • I 1 1 I 

k = 1,  2,  ... 

Error  rates  are  assumed  to  be  constant  and  small,  yielding  a 
vector  of  random  variables. 

T ( t k ) = (tj  (xk) , ...  tN  (tk) ) 

which  are  the  performance  times  associated  with  subtasks  Qj , 
...  Qj^  at  cycles  k=l , 2 , . . . 

I (X)  is  relat  'd  to  T(t)  via  the  information  measure: 

I 

I (1)  II(P)  = - IP.  IP  loa,  p.  . bits 

i 1 j ij  13 

for  the  Markov  process,  and 

t 

(2)  H (P)  = - Ep.  log  P.  bits 

• J ^3 

for  the  non-Markov  process.  Drawing  on  the  results  of  Hick, 
(1952) , the  author  notes  that  the  linear  relationship 

I 1 

tj*  = a + bH(p)  a > 0,  b > 0 

where  t.  . . is  the  decision  time  between  completion 

of  subtask  j-1  and  subtask  j . might  be  expected  to  obtain  for 
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0 < H ( p ) < 3.3  bits  when  ~ 1/L  and  L = 1,  10  for  both 

Markov  and  non-Markov  T (X) . 


I_nj >ut  P ararr.e  t ers 

The  model  requires  specification  of  the  number  of  subtasks, 
Q-j  , the  number  of  decision  alternatives,  and  the  structure 
(Markov  or  non-Markov)  of  the  event  generator  I (X) . 


Model  0 utput 

The  model  predicts  total  performance  ti"  • < n s btasks  as 
a function  of  the  structure  of  I (X) . 


Model  Valid at  Lon 

Though  it  has  been  employed  in  a number  of  s Me  laboratory 
experiments,  there  has  been  no  independent  vili  inn  of  the 
model . 


Comments 

The  formulation  may  be  useful  in  modelling  tasks  requiring 
the  close  c-ordination  of  discrete  motor  movements  in  cases 
where  those  tasks  are  performed  by  skilled  subjects.  An 
interesting  empirical  finding  of  the  study  conducted  in  the 
context  of  this  model  is  that  once  subtasks  are  fully  learned, 
they  can  be  treated  as  separate  entities  for  modelling  purposes. 
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Air-To-Ground  Acquisition  Mod  *ls 


The  reference  cited  below  contains  a valuable  summitry  of 
visual  detection  and  recognition  model s developed  in 
connection  with  military  contracts  over  the  period  1946 
to  1972.  One  or  more  of  these  may  b • useful  in  modelling 
the  acquisition  process  during  the  RL’V  terminal  phase. 

Our  summary  of  this  reference  includes  only  those  aspects 
of  two  of  the  formulations  that  appear  to  have  immediate 
utility  in  that  phase.  The  reader  is  referred  to  the 
original  article  for  fuller  discussion  and  for  pertinent 
comments  relating  to  models  of  this  type. 


Reference 


Greening,  Charles  P.  "Mathematical  modelling  of  air-to- 
ground  target  acquisition,"  Human  Factors,  lq76, 
18(2),  111-148. 


Peso  ; ipb ion 

Target  acquisition  models  di  ffer  in  a number  of  important 
respects.  One  is  the  extent  to  which  they  regard  the  acquisition 
process  as  episodic  — that  is,  consisting  of  st  itist  Lea] Ly 
independent  tasks  such  as  search,  detection,  recognition  — ca- 
ns unitary  and  unanalysable.  A second  difference  relates  to 
whether  or  not  a detection  Jobe  concept  (cf.  Lar.ir,  et  al,  1947  , 
1948)  is  employed  to  model  the  interaction  between  the  geometry 
of  the  search  area  and  the  visual  search  process.  A third 
difference  has  to  do  with  the  characteristics  of  the  search 
itself  --  whether  or  not  .it  is  concerned  with  accju  i >:  i t ion  o' 
targets  which  stand  in  "clear"  (geom-’tr  i c , luminance,  i .}) 
distinction  with  l.he  surroundings  ---  and  with  the  observe:  ' 
knowledge  about  what  ho  is  looking  for  in  ad\  inco. 


1 . MARSAM  I 1 

Of  the  models  employing  a lob  ■ concept  , the  Multiple 
Airb  > rnt  I : lissance  Sensor  Asst  ssi  ■ i 1 M 1 ! ( MAR  5AM  i i ) 

develop  'd  by  Honeywell  is  one  of  the  m > • • cor  glote.  It  is 
modul  e-  and  contain.;,  in  addition  to  submod  •!:;  of  hardware 
sensoi  proc  sses , a ■ 1 iodt  1 ol  t 1 1 1 hui  n ol  ver.  Element 
of  the  latter  arc'  (1)  a search  fund  .ion,  <1  ‘ < • lined  by  size 
of  search  area,  available  time,  tie'  at  1 g I i all  or,  it  ion, 


I 


\ 
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line  of  sight  and  time  varied  conditions;  (2)  a target  detection 
function,  dependent  upon  contrast  and  size  of  target  luminance 
and  "distinctiveness";  (3)  a discrimination  function,  determined 
by  fixcition  area  and  target/non-target  density;  and  (4)  a 
recognition  function,  determined  by  acuity  threshold,  angular 
rate  and  critical  dimension. 

Since  the  functions  are  independent,  the  probability  of 
detection  (Rc])  and  the  conditional  probability  of  recognition 
given  detection  are: 


P . = P, 
d los 

Pds  ‘ 

dl  ‘ Pd3 

and 

II 

r 

, re  sp  o 

v/here , P , 
d 

= probability 

of  det 

■ i ion 

r, 

lOo 

= probabi  itj 
to  me  tar' 

of  the 
■ t 

exisl  ■■  ;e 

Of  ct  lltl 

Pd3 

= probability 
element 

of  fix 

.'.ions  and 

Jw  ; 1 L iip ; 

P?ll 

= probability 

o r d e t 

:tabili  y 

Pd3 

= pr  ib a :>  L J 1 ty 
non-target. 

of  no 

>b  j e e 1 . ' 

.‘onfu.'.i  n ! 

vafv/oon  t. 

P* 

r 

= probab i 1 i ty 

of  r e e 

m i in'  a 

detente  i 

P 

- probabl 
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Where  appropriate,  sub-models  provide  input  to  terms  in 
this  expression.  For  example,  values  of  target  detection 
parameters  are  predicted  on  the  basis  of  an  empirical  function 
derived  from  Blackwell's  (1946)  laboratory  data  on  visual 
contrast  thresholds,  while  recognition  performance  is  predicted 
from  the  model 


= 1-exp 


-k(3/a-3.2)‘ 


f/a  >A-2  ; : 


) oth  ;rwi  se 


where  P~ 

r 


3 

a 


proba b i 1 L t y of  r e c o gn  1 in,-  ta rye t 

characteristic  angular  subtense  of  target 

resolution  capability  of  the  eye  at  riven  apparent 
contrast 


k — a constant 


2.  Rockwell/Autonetecs  Model 

Like  MARSAM  II,  this  model  embodies  an  episodic  view  of  th 
acquisition  process.  Unlike  MARSAM  II,  it  does  not  utilize  a 
visual  lobe  concept.  There  are  two  phases:  (1)  search  and  (2) 

detection  and  recognition.  The  search  process  is  not  supported 
by  a submodel;  instead,  a single  quantity  drawn  from  empirical 
research  is  input.  Submodels  underlying  detection  and  recog- 
nition are,  respectively 


I / 
! / 

P = . •.  I _ I 


C,P)> 


■/(/ 
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i 


where 

a 


C 


C 


T 


probability  of  detection 
resolution  capability  of  eye  at  C = 1 
angular  subtense  of  target 
apparent  contrast 
threshold  contrast 


As  can  be  inferred  by  comparing  these  models,  the  only  difference 
between  detection  and  recognition  is  the  level  of  resolution 
required  by  the  geometry  of  the  target. 

A cumulative  probability  of  detection/recognition  associated 
with  decreasing  range-to-target  can  be  determined  by  summing 
"single  glimpse"  probabilities  of  detect,  on  over  fixation  inter- 
vals available  from  the  time  the  target  first  becomes  visible  to 
the  desired  minimum  range. 

Pcum  = 1 — II  (1-Psg.i) 
i 

where  Pcum  = the  probability  of  detection  after  i independent 
glimpses 

2 

Psgr  = the  single  glimpse  probability  = P^c-(rs/r0)  , 

and  PLc  is  the  probability  of  looking  at  the  target, 
rs  is  the  resolution  capability  of  the  eye,  and 
rQ  is  the  resolution  required  to  detect  or  recognize* 
the  target 


Inpu t Parameters 

The  brevity  of  this  summary  makes  somewhat  inappropriate  a 
listing  of  the  many  inputs  required  for  application  of  the  models. 
In  general,  it  can  be  said  that  information  of  the  following  types 
is  necessary: 

1.  Dimensions  of  target 

2.  Angle  of  visual  axis 

3.  Target/background  contrast 

4.  Visual  contrast  threshold 

5.  Target/background  luminance 

6.  Size  of  visual  lobe  (MARSAM  II  and  other  lobe  models 

on  1 y ) 


A i- 
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7.  Size  of  area  to  be  searched 

8.  Target/background  lumin£ince 

9.  Available  search  time 

10.  Resolving  power  of  eye 


Model  Outputs 

The  major  outputs  of  both  MARS AM  II  and  the  Au tone tics/ 
Rockewell  models  are  (1)  the  probability  of  detecting  a target 
and  (2)  of  recognizing  a target.  In  addition,  the  latter  model 
provides  an  estimate  of  cumulative  probability  of  detection. 


Model  Validation 

As  the  author  points  out,  MARSAM  II,  along  with  the  other 
episodic  models  discussed  in  his  paper,  develops  from  a combin- 
ation of  submodels  that  were,  for  the  most  part,  individually 
validated  during  earlier  investigations.  The  exception  to  this 
generalization  is  the  search  model,  which  remains  unvalidated. 

The  Autonetics  model  has  been  validated,  with  good  results, 
against  simulator  data  in  an  experiment  in  which  recognition 
performance  was  predicted  as  a function  of  physical  range. 


Commer.  ts 


Both  of  the  models  summarized  above,  as  well  as  others  re- 
viewed by  Greening,  appear  to  provide  results  of  some  generality 
across  different  types  of  target  detection  and  recognition 
situations.  As  noted  earlier,  either  or  both  may  be  useful  in 
the  simulation  of  RPV  terminal  phase  operations.  As  the  author 
suggests,  however , major  attention  is  given  to  those  parameter:; 
which  are  reasonably  well  understood  and  rather  loss  attention 
is  given  to  such  things  as  the  character  of  ground  clutter, 
sophistication  of  the  observer,  the  differentiation  oi  multiple 
targets,  etc.  — factors  which  might  be  of  ext  rome  ii  t n :e 
iri  field  conditions.  Also,  it  is  the  case  that  only  the 
acquisition  process  itself  is  of  concern  and  not.  tin  • p.-.  >bl  on 
of  vehicle  control,  terrain  avoidance,  etc.  To  be  used  su  :e 
fully  in  the  simulation  of  a total,  process,  they  must  be  tr  ■ n.ed 
with  trio  or  more  models  that  predict  pc  To:  .unre  on  t:  : n>  other 

dimensions.  Unless  tiies-*  produce  estimate.,  that  are  at  least  as 
a ecu  i ate  us  t.h  e of  the  target  acquis  it  ion  models  am  our  t » be, 
the  pro,  .ion  of  the  latter  and  the  eftort  required  to  specify 
their  input  nay  be  largely  wasted. 


2 r>  8 
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Rouse's  Model,  for  Coen  i_t:  i ve  P fed  i <•:  i on  o r lMLiiro_  f.te  l or. 


Reference 

Rouse,  W.  B.  "A  Model  of  the  Human  in  a Cognitive  Prediction 
Task,"  IEEE  Trans . on  Svs. , Man  and  Cyber.,  Vo].  SMC -3 { 5) , 
1973,  pp.  473-477 . 

Rouse,  W.  B.  "Cognitive  Sources  of  Suboptinal  Human  Prediction," 
Ph.D.  dissertation,  M.T.T.  Cambridge,  Mass.,  Sept.  1972. 


Description 


This  model  was  constructed  for  the  purpose  of  assessing 
human  performance  in  certain  tasks  in  which  events  occur  on  a 
slow-enough  time  scale  so  that  limitations  in  (human  response) 
accuracy  tire  due  more  significantly  to  faulty  memory  and  errors 
in  cognition  than  to  limitations  in  physiological  response. 


The  model  considers  optimal  prediction  of  future  states  of 
discrete  linear  dynamic  systems  given  by: 


XN+1  ' CoYN+1+C  TXm 

N 


where  = 


I 


XN 

n 


the  system  state  at  time  N 


C = 


C2 


a vector  of  constants 


yN  + l 


the  input  a1'  time  N+]  sampled  from  a zero-noan  Gaussian 
process,  and 


o 


a constant 
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For  purposes  of  estimating  a future  state,  it  is  assume'.; 
that  the  human  first  identifies  the  system  and  gains  know  Leu ye 
of  the  input  characteristics  by  collecting  as  data  X^  and 

X and  xv,  XM  - and  x.t  etc.  From  these  data,  he  derives 
an  estimate  of  C which  he  applies  to  his  prediction. 


Critical  in  the  model  is  the  concept  of  information  loss 
over  time.  Two  aspects  of  this  process  are  formally  represented 
with  negative  exponentials:  (1)  a limitation  on  the  amount  of 

data  that  can  be  retained  in  memory;  and  (2)  the  tendency  for 
states  that  have  occurred  recently  to  be  remembered  better  than 
those  that  occurred  at  an  earlier  time. 

A second  important  component  of  its  model  is  the  concept  of 
observation  noise.  It  is  assumed  that  the  standard  deviation 
ox  of  the  human's  estimates  of  a quantity  x are  given  by  the 
equation : 

o 

j—r  - F , a constant 


Input  Parameters 

In  addition  to  system  dynamics,  the  model  requires  input 
of  the  memory  function  (number  of  past  events)  and  o _ . 


Model  Outputs 

The  model  provides  an  estimate  of  the  human's  expectation 
of  C as  a function  of  memory  length  and  observational  accuracy 
Within  the  definition  of  the  model,  this  represents  an  optimal 
output  which  cun  be  compared  against  the  performance  of  real 
sub jee ts . 


. 


L 
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y. - 1 d ol  _Vc 1 1 _id at  ion 

The  author  reports  an  experiment  in  which  subjects  were  required 
to  predict  the  horizontal  displacement  of  the  last  member  of  a 
set  of  eleven  dots  displayed  on  a CRT  as  a function  of  various 
system  dynamics  and  input  variances.  Subjects  both  familiar 
and  unfamiliar  with  the  independent  variables  were  employed. 
Outcomes  accorded  well  with  predictions  of  the  model.  It  was 
also  found  that  changes  in  the  memory  parameter  of  the  model  had 
much  more  influence  on  the  goodness-of-f it  with  experimental  data 
than  did  changes  in  the  observational  noise  parameter. 


Comments 


The  real  utility  of  this  model  is  unknown  at  this  time.  Its 
primary  value  may  be  that  it  provides  a framework  for  assessing 
performance  in  systems  characterized  by  slow,  discrete  changes. 
However,  as  the  author  notes,  its  successful  use  requires  careful 
characterization  of  the  time  scale,  since  others  (e.g.,  Kleinman 
et  al,  1971)  have  found  physiological  limitations  to  be  of  critical 
importance  in  tasks  requiring  rapid  response,  while  still  others 
(e.g.,  Rapaport,  1966;  Sheridan  and  Rouse,  1971)  have  found 
limitations"  in  addition  to  memory  and  observation  noise  in  situations 
where  the  equivalent  of  several  dots  into  the  future  must  be  predicted 
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Pol  lay  1 s Model  of  Dectsi  on  Times  for  Choi  ce  . 
Among  Equally  Att rae fcive  A1 1 er n atives 


R *f eren : es 


Pollay,  R.  VI,  "A  mode  I of  decision  times  In  difficult  decisl  n 
situation  a , " Pry  . ‘ . Rev . , 1970,  77(iO,  27 --283  . 

Hendrick,  C.  , '•'.ills,  J.  and  Kiesler,  C.  A.  "Decision  time  as 
a functic  i of  the  number  and  complexity  of  equally  a tern  si  L\ 
alternatives,"  J.  A : • s . arid  Doe.  Psychol.,  1968,  8,  313-318. 

Kiesler,  C . A.  "Conflict  arid  the  number  of  choice  alternat  i s," 
Psychol.  Ren.,  196c,  18,  603-6] 0. 


Descript  Ion 

This  model  was  developed  to  predict  the  apparently  anomalous 
finding  of  Hendrick,  et  al  (1968)  and  Kiesler  (1966)  that  decision 
makers  took  less  time  when  all  four  members  of  a set  of  alterna- 
tives were  equally  attractive  than  when  two  of  the  members  wore 
clearly  inferior  and  easily  rej eatable.  The  model  requires  th 
fol  lowi ng  assur.pt. i ons  : 
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Description 

This  model  was  developed  for  the  purpose  of  estimating  work- 
load in  tasks  where  the  operator  appears  to  be  able  to  share  time 
between  continuous  and  discontinuous  subtasks.  The  author  makes 
the  point  that  workload  associated  with  such  tasks  may  be  very 
much  overestimated  by  models  that  view  the  operator  as  an 
essentially  single-channel  device,  particularly  when  he  is  well- 
trained  and  highly  experienced. 

The  basic  idea  of  the  Function  Interlace  Model  is  expressed 
by  the  equation. 


where  mi  and  u> 2 = the  instantaneous  workloads  predicted  for 
subtask  1 and  subtask  2,  respectively;  and  I = the  interlace 
coefficient  associated  with  the  combined  subtasks. 

To  use  the  model  one  must  first  derive  estimates  of  the 
workloads  associated  with  the  subtasks  when  each  is  performed 
independently.  By  analysis  or  by  si:,  illation  techniques  employing 
a tertiary  task  designed  to  provide  maximum  interference  with  the 
primary  subtasks,  the  interlace  coefficient  is  then  estimat  -d. 


Inpu t_  Parameters 

Estimates  of  workload  associated  with  each  subtnsk  ind  an 
estimate  of  the  interlace  coefficient  are  required. 
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Model  Output 

The  model  provides  an  estimate  o£  workload  in  tasks  permit- 
ting parallel  processing. 


Model  Validation 

The  model  is  still  in  evaluation  and  has  not  yet  undergone 
complete  validation.  Primary  attention  has  been  focussed  on 
validating  the  techniques  (e.g.,  eye  movements)  for  assessment 
of  workload  of  independent  subtasks. 


Comments 

This  model  appears  to  require  a good  deal  of  artistry  during 
assessment  of  independent  and  interactive  workload,  particularly 
where  simulation  is  not  a viable  method  for  establi  ;hing  baseline 
data.  Also,  there  is  a question  as  to  how  interlace  effects  can 
be  estimated  for  any  group  of  subtasks  without  making  the 
equivalent  of  a complete  factorial  analysis  in  which  each  subtask 
is  combined  with  each  other.  If  this  need  be  done,  one  wonders 
about  the  justification  for  considering  "Function  Interlace"  to 
be  a model.  We  suspect  that  a major  value  in  viewing  operator 
workload  in  this  framework  is  that  one  is  forced  to  pay  vary 
close  attention  to  the  structural  requirements  and  the  inputs 
and  outputs  of  the  subtask  sequence  - hence,  to  become  in  Lima t lv 
familiar  with  the  demands  made  upon  the  operator.  Perhaps,  v > th 
this  familiarity,  the  assessment  of  workload  is  quite  straight- 
forward and  of  high  reliability. 
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